2024-04-25T20:41:41.181 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T20:41:41.181 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653 --owner scheduled_leonidus@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --name leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi -- /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/orig.config.yaml 2024-04-25T20:41:41.240 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T20:41:41.330 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7673653' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: e221b808d285b9d02e54cc10c53c84188e8e41e3 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2919 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: e221b808d285b9d02e54cc10c53c84188e8e41e3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGLTAP5SraWWthZaMaHi9MKtXXMliswmWAQwrSWAgNKuEKB4+VTxdfXb8uSZN5fPZzZoI+LHF2xoxXemM4QCJ/8= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMdypklKpkF+hjw/7UuCw/DBjZAoQ/++7VZj7uXlAK+x39lQ/dHVYatCVBYL9aroC0A/3sN3fHAGcPzm/TlgYBw= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCejTNE7xCNe5W40Lmyc6UVh8tiwuJcqn0Vk53uxWfqsD/vJDU+o/ZpZVlklEuiozQR9HAz9Pu1eDOxH6cZHgx0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_19:59:35 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T20:41:41.331 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa; will attempt to use it 2024-04-25T20:41:41.331 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks 2024-04-25T20:41:41.331 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T20:41:41.333 INFO:teuthology.task.internal:Checking packages... 2024-04-25T20:41:41.355 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T20:41:41.355 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T20:41:41.355 INFO:teuthology.packaging:ref: None 2024-04-25T20:41:41.356 INFO:teuthology.packaging:tag: None 2024-04-25T20:41:41.356 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T20:41:41.356 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:41:41.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T20:41:41.514 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T20:41:41.515 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T20:41:41.522 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T20:41:41.522 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T20:41:41.563 INFO:teuthology.task.internal:Saving configuration 2024-04-25T20:41:41.583 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T20:41:41.588 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T20:41:41.611 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:36:32.507325', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGLTAP5SraWWthZaMaHi9MKtXXMliswmWAQwrSWAgNKuEKB4+VTxdfXb8uSZN5fPZzZoI+LHF2xoxXemM4QCJ/8='} 2024-04-25T20:41:41.634 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:36:32.508597', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMdypklKpkF+hjw/7UuCw/DBjZAoQ/++7VZj7uXlAK+x39lQ/dHVYatCVBYL9aroC0A/3sN3fHAGcPzm/TlgYBw='} 2024-04-25T20:41:41.657 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:36:32.505722', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCejTNE7xCNe5W40Lmyc6UVh8tiwuJcqn0Vk53uxWfqsD/vJDU+o/ZpZVlklEuiozQR9HAz9Pu1eDOxH6cZHgx0='} 2024-04-25T20:41:41.657 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T20:41:41.663 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T20:41:41.663 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T20:41:41.664 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T20:41:41.664 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T20:41:41.771 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc41e43e0d0>, signals=[15]) 2024-04-25T20:41:41.772 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T20:41:41.779 INFO:teuthology.task.internal:Opening connections... 2024-04-25T20:41:41.779 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T20:41:41.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:41:41.855 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-04-25T20:41:41.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:41:41.926 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T20:41:41.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:41:41.998 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T20:41:42.005 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-04-25T20:41:42.023 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-04-25T20:41:42.023 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-04-25T20:41:42.081 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-04-25T20:41:42.081 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-04-25T20:41:42.081 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-04-25T20:41:42.081 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:41:42.082 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:41:42.083 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-04-25T20:41:42.107 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-04-25T20:41:42.124 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-04-25T20:41:42.124 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:41:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:41:42.182 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:41:42.182 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:41:42.182 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-04-25T20:41:42.207 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-04-25T20:41:42.224 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-04-25T20:41:42.224 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-04-25T20:41:42.281 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:41:42.282 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:41:42.283 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:41:42.283 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-04-25T20:41:42.307 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T20:41:42.317 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T20:41:42.324 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T20:41:42.324 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:41:42.326 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:41:42.329 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:41:42.344 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T20:41:42.350 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T20:41:42.350 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:41:42.385 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:41:42.387 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:41:42.476 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T20:41:42.552 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T20:41:42.567 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T20:41:42.567 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T20:41:42.567 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T20:41:42.568 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:41:42.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:41:42.585 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-25T20:41:42.644 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-437.el9.x86_64 2024-04-25T20:41:42.644 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.14.0-437.el9.x86_64 2024-04-25T20:41:42.644 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-04-25T20:41:44.900 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T20:41:46.909 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T20:41:51.160 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T20:41:52.575 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 16 kB 00:00 2024-04-25T20:41:53.643 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-25T20:41:58.909 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-25T20:42:00.824 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T20:42:00.824 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T20:42:00.924 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-25T20:42:00.925 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-25T20:42:00.925 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-25T20:42:01.040 DEBUG:teuthology.orchestra.run.smithi047:> echo no | sudo yum reinstall kernel || true 2024-04-25T20:42:01.600 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:41:58 PM UTC. 2024-04-25T20:42:01.724 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout:Reinstalling: 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:42:01.734 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 255 k 2024-04-25T20:42:01.735 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 0 2024-04-25T20:42:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:Operation aborted. 2024-04-25T20:42:01.787 INFO:teuthology.orchestra.run.smithi047.stdout:Is this ok [y/N]: 2024-04-25T20:42:01.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum reinstall -y kernel || true 2024-04-25T20:42:02.304 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 08:41:58 PM UTC. 2024-04-25T20:42:02.424 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-25T20:42:02.425 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-25T20:42:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-04-25T20:42:02.425 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-25T20:42:02.425 INFO:teuthology.orchestra.run.smithi047.stdout:Reinstalling: 2024-04-25T20:42:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:42:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:42:02.426 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-25T20:42:02.426 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-25T20:42:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:42:02.426 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 255 k 2024-04-25T20:42:02.426 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 0 2024-04-25T20:42:02.426 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-25T20:42:02.686 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-25T20:42:02.686 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:42:02.687 INFO:teuthology.orchestra.run.smithi047.stdout:Total 976 kB/s | 255 kB 00:00 2024-04-25T20:42:02.710 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-25T20:42:02.893 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-25T20:42:02.893 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-25T20:42:02.898 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-25T20:42:02.898 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-25T20:42:03.021 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-25T20:42:03.120 INFO:teuthology.orchestra.run.smithi047.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:42:03.121 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:42:03.753 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:42:03.753 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:42:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:42:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:42:04.498 INFO:teuthology.orchestra.run.smithi047.stdout:Reinstalled: 2024-04-25T20:42:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T20:42:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:42:04.498 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-25T20:42:04.666 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T20:42:04.714 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T20:42:04.715 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T20:42:04.715 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T20:42:04.715 DEBUG:teuthology.parallel:result is None 2024-04-25T20:42:04.715 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T20:42:04.723 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T20:42:04.723 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:42:04.725 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:42:04.728 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:42:04.745 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T20:42:04.752 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T20:42:04.760 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T20:42:04.760 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:42:04.781 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:42:04.786 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:42:04.898 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T20:42:04.904 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T20:42:04.904 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:42:04.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:42:04.921 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:42:04.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:42:04.938 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:42:04.940 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:42:04.964 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:42:04.969 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:42:04.990 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:42:04.999 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:42:05.022 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:42:05.026 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:42:05.053 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:42:05.055 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T20:42:05.062 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T20:42:05.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:42:05.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:42:05.070 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:42:05.096 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T20:42:05.107 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T20:42:05.107 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:42:05.130 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:42:05.138 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:42:05.154 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:42:05.220 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:42:05.272 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:42:05.349 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:42:05.393 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:42:05.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:42:05.457 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:42:05.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:42:05.553 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:42:05.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:42:05.679 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:42:05.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:42:05.747 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:42:05.790 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:42:05.840 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:42:05.923 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:42:05.964 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:42:05.964 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:42:06.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-04-25T20:42:06.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-04-25T20:42:06.034 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-04-25T20:42:06.069 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:42:06.070 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:42:06.101 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:42:06.568 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T20:42:06.576 INFO:teuthology.task.internal:Starting timer... 2024-04-25T20:42:06.576 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T20:42:06.585 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T20:42:06.592 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T20:42:06.593 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-04-25T20:42:06.672 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-04-25T20:42:06.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-04-25T20:42:06.737 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-04-25T20:42:06.739 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-04-25T20:42:06.803 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-04-25T20:42:06.805 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T20:42:06.805 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-04-25T20:42:06.824 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-04-25T20:42:06.824 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-04-25T20:42:06.846 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-04-25T20:42:06.846 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-04-25T20:42:06.872 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-04-25T20:42:06.872 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-04-25T20:42:06.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:42:06.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:42:06.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:42:06.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:42:06.924 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:42:06.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:42:06.951 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T20:42:06.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:42:06.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:42:07.010 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:42:07.036 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T20:42:07.046 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T20:42:07.063 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T20:42:07.064 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi092.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T20:45:52.969 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-04-25T20:45:52.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-25T20:45:52.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:45:53.050 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-25T20:45:53.126 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-25T20:45:53.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-04-25T20:45:53.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:45:53.199 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-04-25T20:45:53.277 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-04-25T20:45:53.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-25T20:45:53.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:45:53.351 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-04-25T20:45:53.430 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-25T20:45:53.430 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T20:45:53.443 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T20:45:53.443 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:45:53.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:45:53.446 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:45:53.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:45:53.448 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:45:53.448 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:45:53.476 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:45:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:45:53.490 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:45:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:45:53.499 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:45:53.513 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:45:53.513 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-04-25T20:45:53.518 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-04-25T20:45:53.524 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-04-25T20:45:53.530 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-04-25T20:45:53.536 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-04-25T20:45:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:45:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:45:53.546 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-04-25T20:45:53.550 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:45:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:45:53.559 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:45:53.572 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:45:53.602 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:45:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:45:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-04-25T20:45:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.613 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:45:53.617 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:45:53.617 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-25T20:45:53.617 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.617 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.617 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.617 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.626 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:45:53.629 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:45:53.629 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-25T20:45:53.629 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.629 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.629 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.630 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:45:53.630 INFO:teuthology.run_tasks:Running task install... 2024-04-25T20:45:53.639 DEBUG:teuthology.task.install:project ceph 2024-04-25T20:45:53.639 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T20:45:53.639 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T20:45:53.639 INFO:teuthology.task.install:Using flavor: default 2024-04-25T20:45:53.645 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T20:45:53.645 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T20:45:53.645 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:45:53.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:53.648 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:45:53.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:53.649 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:45:53.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:53.796 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:45:53.796 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:45:53.871 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:45:53.871 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:45:53.872 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:45:53.872 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:45:54.105 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:45:54.105 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:45:54.105 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:45:54.115 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:45:54.116 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:45:54.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:45:54.131 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:45:54.131 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:45:54.131 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:45:54.132 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:45:54.132 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:45:54.141 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:45:54.142 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:45:54.160 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:45:54.160 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:45:54.199 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:45:54.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:45:54.227 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:45:54.273 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:45:54.279 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:45:54.304 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:45:54.336 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-04-25T20:45:54.338 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-04-25T20:45:54.342 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-04-25T20:45:54.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-04-25T20:45:54.369 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-04-25T20:45:54.371 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-04-25T20:45:54.564 INFO:teuthology.orchestra.run.smithi186.stdout:48 files removed 2024-04-25T20:45:54.568 INFO:teuthology.orchestra.run.smithi092.stdout:48 files removed 2024-04-25T20:45:54.592 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:45:54.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:45:54.607 INFO:teuthology.orchestra.run.smithi047.stdout:48 files removed 2024-04-25T20:45:54.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:45:55.303 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 185 kB/s | 83 kB 00:00 2024-04-25T20:45:55.353 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 168 kB/s | 83 kB 00:00 2024-04-25T20:45:55.480 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 146 kB/s | 83 kB 00:00 2024-04-25T20:45:55.787 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-25T20:45:55.870 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-25T20:45:55.973 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-25T20:45:56.179 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-25T20:45:56.254 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-25T20:45:56.431 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-25T20:45:56.757 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T20:45:56.940 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T20:45:57.194 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T20:45:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-25T20:45:58.774 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-25T20:45:59.154 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T20:46:02.717 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T20:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-25T20:46:03.613 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.2 MB 00:00 2024-04-25T20:46:04.148 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-25T20:46:04.631 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T20:46:05.133 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-25T20:46:06.111 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-25T20:46:06.735 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T20:46:08.323 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-25T20:46:10.108 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-25T20:46:11.782 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-25T20:46:12.260 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:46:12.260 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:46:12.272 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:46:12.272 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:46:12.273 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:46:12.274 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:46:12.275 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:46:12.278 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:46:12.278 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:46:12.340 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-25T20:46:12.360 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-25T20:46:12.360 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-25T20:46:12.360 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-25T20:46:12.360 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-25T20:46:12.360 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:46:12.360 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:46:12.360 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:46:12.360 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:46:12.361 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:46:12.362 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:46:12.364 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:46:12.365 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:46:12.366 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:46:12.367 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:46:12.368 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:46:12.369 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:46:12.370 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:46:12.371 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout:Installing weak dependencies: 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout:Install 148 Packages 2024-04-25T20:46:12.372 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 4 Packages 2024-04-25T20:46:12.373 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:46:12.373 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 222 M 2024-04-25T20:46:12.373 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-25T20:46:13.185 INFO:teuthology.orchestra.run.smithi186.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-25T20:46:13.493 INFO:teuthology.orchestra.run.smithi186.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 890 kB 00:00 2024-04-25T20:46:13.571 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-25T20:46:13.577 INFO:teuthology.orchestra.run.smithi186.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-04-25T20:46:13.660 INFO:teuthology.orchestra.run.smithi186.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 7.4 MB/s | 5.4 MB 00:00 2024-04-25T20:46:13.844 INFO:teuthology.orchestra.run.smithi186.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 8.2 MB/s | 1.5 MB 00:00 2024-04-25T20:46:13.916 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:46:13.917 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:46:13.927 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:46:13.928 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:46:13.929 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:46:13.930 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:46:13.933 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:46:13.934 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:46:13.934 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:46:13.994 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-25T20:46:14.009 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-25T20:46:14.009 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-25T20:46:14.009 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-25T20:46:14.009 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-25T20:46:14.009 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:46:14.010 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:46:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:46:14.012 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:46:14.013 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:46:14.014 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:46:14.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:46:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:46:14.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:46:14.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:46:14.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:46:14.020 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout:Install 148 Packages 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 4 Packages 2024-04-25T20:46:14.021 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:46:14.022 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 222 M 2024-04-25T20:46:14.022 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-25T20:46:14.094 INFO:teuthology.orchestra.run.smithi186.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 4.6 MB/s | 2.4 MB 00:00 2024-04-25T20:46:14.361 INFO:teuthology.orchestra.run.smithi186.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 9.1 MB/s | 4.7 MB 00:00 2024-04-25T20:46:14.783 INFO:teuthology.orchestra.run.smithi092.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 11 MB/s | 21 MB 00:01 2024-04-25T20:46:14.995 INFO:teuthology.orchestra.run.smithi186.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 303 kB/s | 25 kB 00:00 2024-04-25T20:46:15.603 INFO:teuthology.orchestra.run.smithi186.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 8.5 MB/s | 11 MB 00:01 2024-04-25T20:46:15.617 INFO:teuthology.orchestra.run.smithi092.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 890 kB 00:00 2024-04-25T20:46:15.679 INFO:teuthology.orchestra.run.smithi186.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 225 kB 00:00 2024-04-25T20:46:15.733 INFO:teuthology.orchestra.run.smithi092.stdout:(3/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.2 MB/s | 5.4 MB 00:01 2024-04-25T20:46:15.737 INFO:teuthology.orchestra.run.smithi186.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 464 kB/s | 27 kB 00:00 2024-04-25T20:46:15.775 INFO:teuthology.orchestra.run.smithi092.stdout:(4/152): ceph-immutable-object-cache-19.0.0-315 915 kB/s | 145 kB 00:00 2024-04-25T20:46:15.813 INFO:teuthology.orchestra.run.smithi186.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 423 kB/s | 32 kB 00:00 2024-04-25T20:46:15.839 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:46:15.840 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:46:15.851 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:46:15.851 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:46:15.852 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:46:15.853 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:46:15.855 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:46:15.857 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:46:15.858 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:46:15.922 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-25T20:46:15.931 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-25T20:46:15.931 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-25T20:46:15.931 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-25T20:46:15.931 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-25T20:46:15.931 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:46:15.932 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:46:15.933 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:46:15.934 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:46:15.935 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:46:15.936 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:46:15.937 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:46:15.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:46:15.939 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:46:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:46:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:46:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-25T20:46:15.943 INFO:teuthology.orchestra.run.smithi047.stdout:Install 148 Packages 2024-04-25T20:46:15.944 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 4 Packages 2024-04-25T20:46:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:46:15.944 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 222 M 2024-04-25T20:46:15.944 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-25T20:46:15.996 INFO:teuthology.orchestra.run.smithi186.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.0 MB/s | 747 kB 00:00 2024-04-25T20:46:16.063 INFO:teuthology.orchestra.run.smithi186.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 163 kB 00:00 2024-04-25T20:46:16.075 INFO:teuthology.orchestra.run.smithi092.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 6.9 MB/s | 2.4 MB 00:00 2024-04-25T20:46:16.146 INFO:teuthology.orchestra.run.smithi186.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-25T20:46:16.230 INFO:teuthology.orchestra.run.smithi186.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.9 MB/s | 500 kB 00:00 2024-04-25T20:46:16.492 INFO:teuthology.orchestra.run.smithi092.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.1 MB/s | 1.5 MB 00:00 2024-04-25T20:46:16.889 INFO:teuthology.orchestra.run.smithi186.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 8.1 MB/s | 5.4 MB 00:00 2024-04-25T20:46:16.901 INFO:teuthology.orchestra.run.smithi092.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.7 MB/s | 4.7 MB 00:00 2024-04-25T20:46:16.939 INFO:teuthology.orchestra.run.smithi186.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 907 kB/s | 45 kB 00:00 2024-04-25T20:46:16.947 INFO:teuthology.orchestra.run.smithi047.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-25T20:46:17.039 INFO:teuthology.orchestra.run.smithi186.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-25T20:46:17.123 INFO:teuthology.orchestra.run.smithi186.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.9 MB/s | 161 kB 00:00 2024-04-25T20:46:17.215 INFO:teuthology.orchestra.run.smithi186.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.4 MB/s | 320 kB 00:00 2024-04-25T20:46:17.315 INFO:teuthology.orchestra.run.smithi186.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 298 kB 00:00 2024-04-25T20:46:17.408 INFO:teuthology.orchestra.run.smithi186.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.1 MB/s | 99 kB 00:00 2024-04-25T20:46:17.500 INFO:teuthology.orchestra.run.smithi186.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 931 kB/s | 85 kB 00:00 2024-04-25T20:46:18.031 INFO:teuthology.orchestra.run.smithi047.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 821 kB/s | 890 kB 00:01 2024-04-25T20:46:18.033 INFO:teuthology.orchestra.run.smithi186.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 5.8 MB/s | 3.1 MB 00:00 2024-04-25T20:46:18.117 INFO:teuthology.orchestra.run.smithi186.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T20:46:18.173 INFO:teuthology.orchestra.run.smithi047.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.0 MB/s | 145 kB 00:00 2024-04-25T20:46:18.183 INFO:teuthology.orchestra.run.smithi186.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 513 kB/s | 34 kB 00:00 2024-04-25T20:46:18.259 INFO:teuthology.orchestra.run.smithi186.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.9 MB/s | 142 kB 00:00 2024-04-25T20:46:18.751 INFO:teuthology.orchestra.run.smithi092.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 5.7 MB/s | 11 MB 00:01 2024-04-25T20:46:18.826 INFO:teuthology.orchestra.run.smithi092.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 337 kB/s | 25 kB 00:00 2024-04-25T20:46:18.842 INFO:teuthology.orchestra.run.smithi186.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 6.1 MB/s | 3.5 MB 00:00 2024-04-25T20:46:19.406 INFO:teuthology.orchestra.run.smithi047.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-25T20:46:19.509 INFO:teuthology.orchestra.run.smithi186.stdout:(31/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 3.1 MB/s | 17 MB 00:05 2024-04-25T20:46:19.701 INFO:teuthology.orchestra.run.smithi186.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.3 MB/s | 249 kB 00:00 2024-04-25T20:46:19.809 INFO:teuthology.orchestra.run.smithi186.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 455 kB/s | 49 kB 00:00 2024-04-25T20:46:19.832 INFO:teuthology.orchestra.run.smithi047.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.7 MB/s | 5.4 MB 00:03 2024-04-25T20:46:19.902 INFO:teuthology.orchestra.run.smithi186.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 179 kB/s | 16 kB 00:00 2024-04-25T20:46:19.940 INFO:teuthology.orchestra.run.smithi047.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-25T20:46:20.096 INFO:teuthology.orchestra.run.smithi186.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.4 MB/s | 273 kB 00:00 2024-04-25T20:46:20.254 INFO:teuthology.orchestra.run.smithi186.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-04-25T20:46:20.346 INFO:teuthology.orchestra.run.smithi186.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 199 kB/s | 18 kB 00:00 2024-04-25T20:46:20.446 INFO:teuthology.orchestra.run.smithi186.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 804 kB/s | 80 kB 00:00 2024-04-25T20:46:20.497 INFO:teuthology.orchestra.run.smithi186.stdout:(39/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.9 MB/s | 762 kB 00:00 2024-04-25T20:46:20.539 INFO:teuthology.orchestra.run.smithi186.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.5 MB/s | 803 kB 00:00 2024-04-25T20:46:20.564 INFO:teuthology.orchestra.run.smithi186.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-25T20:46:20.598 INFO:teuthology.orchestra.run.smithi186.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-25T20:46:20.631 INFO:teuthology.orchestra.run.smithi186.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T20:46:20.656 INFO:teuthology.orchestra.run.smithi186.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T20:46:20.690 INFO:teuthology.orchestra.run.smithi186.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T20:46:20.715 INFO:teuthology.orchestra.run.smithi186.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T20:46:20.740 INFO:teuthology.orchestra.run.smithi186.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T20:46:20.766 INFO:teuthology.orchestra.run.smithi186.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T20:46:20.799 INFO:teuthology.orchestra.run.smithi186.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T20:46:20.825 INFO:teuthology.orchestra.run.smithi186.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T20:46:20.842 INFO:teuthology.orchestra.run.smithi186.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 515 kB/s | 21 kB 00:00 2024-04-25T20:46:20.848 INFO:teuthology.orchestra.run.smithi047.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 4.6 MB/s | 4.7 MB 00:01 2024-04-25T20:46:20.867 INFO:teuthology.orchestra.run.smithi186.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-04-25T20:46:20.883 INFO:teuthology.orchestra.run.smithi186.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 794 kB/s | 33 kB 00:00 2024-04-25T20:46:20.959 INFO:teuthology.orchestra.run.smithi186.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-25T20:46:20.976 INFO:teuthology.orchestra.run.smithi186.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 200 kB/s | 18 kB 00:00 2024-04-25T20:46:20.993 INFO:teuthology.orchestra.run.smithi186.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-04-25T20:46:21.018 INFO:teuthology.orchestra.run.smithi186.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T20:46:21.043 INFO:teuthology.orchestra.run.smithi186.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T20:46:21.102 INFO:teuthology.orchestra.run.smithi186.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T20:46:21.127 INFO:teuthology.orchestra.run.smithi186.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 582 kB 00:00 2024-04-25T20:46:21.153 INFO:teuthology.orchestra.run.smithi186.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 515 kB/s | 25 kB 00:00 2024-04-25T20:46:21.178 INFO:teuthology.orchestra.run.smithi186.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T20:46:21.203 INFO:teuthology.orchestra.run.smithi186.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T20:46:21.228 INFO:teuthology.orchestra.run.smithi186.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T20:46:21.262 INFO:teuthology.orchestra.run.smithi186.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-25T20:46:21.354 INFO:teuthology.orchestra.run.smithi186.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-25T20:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 3.4 MB/s | 17 MB 00:04 2024-04-25T20:46:21.429 INFO:teuthology.orchestra.run.smithi186.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-25T20:46:21.471 INFO:teuthology.orchestra.run.smithi186.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T20:46:21.505 INFO:teuthology.orchestra.run.smithi186.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T20:46:21.530 INFO:teuthology.orchestra.run.smithi186.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T20:46:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.1 MB/s | 225 kB 00:00 2024-04-25T20:46:21.664 INFO:teuthology.orchestra.run.smithi186.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 37 MB/s | 5.8 MB 00:00 2024-04-25T20:46:21.697 INFO:teuthology.orchestra.run.smithi186.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-25T20:46:21.723 INFO:teuthology.orchestra.run.smithi186.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T20:46:21.751 INFO:teuthology.orchestra.run.smithi092.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 232 kB/s | 27 kB 00:00 2024-04-25T20:46:21.840 INFO:teuthology.orchestra.run.smithi186.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-04-25T20:46:21.860 INFO:teuthology.orchestra.run.smithi092.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 293 kB/s | 32 kB 00:00 2024-04-25T20:46:21.865 INFO:teuthology.orchestra.run.smithi186.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-04-25T20:46:21.890 INFO:teuthology.orchestra.run.smithi186.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T20:46:21.915 INFO:teuthology.orchestra.run.smithi186.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-04-25T20:46:21.941 INFO:teuthology.orchestra.run.smithi186.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T20:46:21.966 INFO:teuthology.orchestra.run.smithi186.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T20:46:21.991 INFO:teuthology.orchestra.run.smithi186.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-04-25T20:46:22.092 INFO:teuthology.orchestra.run.smithi186.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-04-25T20:46:22.117 INFO:teuthology.orchestra.run.smithi186.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-04-25T20:46:22.142 INFO:teuthology.orchestra.run.smithi186.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T20:46:22.152 INFO:teuthology.orchestra.run.smithi092.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.5 MB/s | 747 kB 00:00 2024-04-25T20:46:22.167 INFO:teuthology.orchestra.run.smithi186.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T20:46:22.184 INFO:teuthology.orchestra.run.smithi186.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T20:46:22.209 INFO:teuthology.orchestra.run.smithi186.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-04-25T20:46:22.235 INFO:teuthology.orchestra.run.smithi186.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T20:46:22.260 INFO:teuthology.orchestra.run.smithi186.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T20:46:22.261 INFO:teuthology.orchestra.run.smithi092.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-25T20:46:22.277 INFO:teuthology.orchestra.run.smithi186.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T20:46:22.294 INFO:teuthology.orchestra.run.smithi186.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-25T20:46:22.386 INFO:teuthology.orchestra.run.smithi092.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.0 MB/s | 126 kB 00:00 2024-04-25T20:46:22.419 INFO:teuthology.orchestra.run.smithi186.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 336 kB/s | 42 kB 00:00 2024-04-25T20:46:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 6.7 MB/s | 17 MB 00:02 2024-04-25T20:46:22.557 INFO:teuthology.orchestra.run.smithi047.stdout:(9/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 6.2 MB/s | 11 MB 00:01 2024-04-25T20:46:22.582 INFO:teuthology.orchestra.run.smithi047.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 189 kB/s | 25 kB 00:00 2024-04-25T20:46:22.611 INFO:teuthology.orchestra.run.smithi186.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-04-25T20:46:22.619 INFO:teuthology.orchestra.run.smithi092.stdout:(17/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.6 MB/s | 21 MB 00:08 2024-04-25T20:46:22.628 INFO:teuthology.orchestra.run.smithi186.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 214 kB/s | 44 kB 00:00 2024-04-25T20:46:22.649 INFO:teuthology.orchestra.run.smithi047.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.3 MB/s | 225 kB 00:00 2024-04-25T20:46:22.653 INFO:teuthology.orchestra.run.smithi092.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 1.8 MB/s | 500 kB 00:00 2024-04-25T20:46:22.653 INFO:teuthology.orchestra.run.smithi186.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-25T20:46:22.687 INFO:teuthology.orchestra.run.smithi186.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-04-25T20:46:22.713 INFO:teuthology.orchestra.run.smithi186.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-04-25T20:46:22.726 INFO:teuthology.orchestra.run.smithi047.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 360 kB/s | 27 kB 00:00 2024-04-25T20:46:22.738 INFO:teuthology.orchestra.run.smithi186.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T20:46:22.753 INFO:teuthology.orchestra.run.smithi092.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 452 kB/s | 45 kB 00:00 2024-04-25T20:46:22.763 INFO:teuthology.orchestra.run.smithi186.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T20:46:22.788 INFO:teuthology.orchestra.run.smithi186.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T20:46:22.808 INFO:teuthology.orchestra.run.smithi047.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 381 kB/s | 32 kB 00:00 2024-04-25T20:46:22.813 INFO:teuthology.orchestra.run.smithi186.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T20:46:22.839 INFO:teuthology.orchestra.run.smithi186.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-25T20:46:22.845 INFO:teuthology.orchestra.run.smithi092.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T20:46:22.864 INFO:teuthology.orchestra.run.smithi186.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-04-25T20:46:22.898 INFO:teuthology.orchestra.run.smithi186.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T20:46:22.962 INFO:teuthology.orchestra.run.smithi092.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.4 MB/s | 161 kB 00:00 2024-04-25T20:46:22.967 INFO:teuthology.orchestra.run.smithi047.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.6 MB/s | 747 kB 00:00 2024-04-25T20:46:23.067 INFO:teuthology.orchestra.run.smithi047.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 163 kB 00:00 2024-04-25T20:46:23.104 INFO:teuthology.orchestra.run.smithi092.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 320 kB 00:00 2024-04-25T20:46:23.150 INFO:teuthology.orchestra.run.smithi047.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-25T20:46:23.263 INFO:teuthology.orchestra.run.smithi092.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 298 kB 00:00 2024-04-25T20:46:23.267 INFO:teuthology.orchestra.run.smithi047.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 4.2 MB/s | 500 kB 00:00 2024-04-25T20:46:23.330 INFO:teuthology.orchestra.run.smithi092.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.5 MB/s | 99 kB 00:00 2024-04-25T20:46:23.430 INFO:teuthology.orchestra.run.smithi092.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 854 kB/s | 85 kB 00:00 2024-04-25T20:46:23.773 INFO:teuthology.orchestra.run.smithi186.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 353 kB/s | 309 kB 00:00 2024-04-25T20:46:23.873 INFO:teuthology.orchestra.run.smithi186.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-25T20:46:24.001 INFO:teuthology.orchestra.run.smithi047.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.3 MB/s | 5.4 MB 00:00 2024-04-25T20:46:24.168 INFO:teuthology.orchestra.run.smithi047.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 271 kB/s | 45 kB 00:00 2024-04-25T20:46:24.252 INFO:teuthology.orchestra.run.smithi047.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.6 MB/s | 135 kB 00:00 2024-04-25T20:46:24.297 INFO:teuthology.orchestra.run.smithi092.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.6 MB/s | 3.1 MB 00:00 2024-04-25T20:46:24.327 INFO:teuthology.orchestra.run.smithi047.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.1 MB/s | 161 kB 00:00 2024-04-25T20:46:24.380 INFO:teuthology.orchestra.run.smithi092.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T20:46:24.427 INFO:teuthology.orchestra.run.smithi047.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.1 MB/s | 320 kB 00:00 2024-04-25T20:46:24.456 INFO:teuthology.orchestra.run.smithi092.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 456 kB/s | 34 kB 00:00 2024-04-25T20:46:24.473 INFO:teuthology.orchestra.run.smithi186.stdout:(106/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 5.2 MB/s | 49 MB 00:09 2024-04-25T20:46:24.498 INFO:teuthology.orchestra.run.smithi186.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 336 kB/s | 548 kB 00:01 2024-04-25T20:46:24.519 INFO:teuthology.orchestra.run.smithi047.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 298 kB 00:00 2024-04-25T20:46:24.539 INFO:teuthology.orchestra.run.smithi092.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.7 MB/s | 142 kB 00:00 2024-04-25T20:46:24.574 INFO:teuthology.orchestra.run.smithi186.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-25T20:46:24.602 INFO:teuthology.orchestra.run.smithi047.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-25T20:46:24.686 INFO:teuthology.orchestra.run.smithi047.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T20:46:24.758 INFO:teuthology.orchestra.run.smithi186.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-25T20:46:25.014 INFO:teuthology.orchestra.run.smithi092.stdout:(30/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 5.4 MB 00:02 2024-04-25T20:46:25.033 INFO:teuthology.orchestra.run.smithi186.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 550 kB/s | 151 kB 00:00 2024-04-25T20:46:25.058 INFO:teuthology.orchestra.run.smithi186.stdout:(111/152): liboath-2.6.7-2.el9.x86_64.rpm 87 kB/s | 49 kB 00:00 2024-04-25T20:46:25.078 INFO:teuthology.orchestra.run.smithi047.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.9 MB/s | 3.1 MB 00:00 2024-04-25T20:46:25.170 INFO:teuthology.orchestra.run.smithi047.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 170 kB 00:00 2024-04-25T20:46:25.246 INFO:teuthology.orchestra.run.smithi047.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 455 kB/s | 34 kB 00:00 2024-04-25T20:46:25.338 INFO:teuthology.orchestra.run.smithi047.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-25T20:46:25.474 INFO:teuthology.orchestra.run.smithi092.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.8 MB/s | 3.5 MB 00:00 2024-04-25T20:46:25.582 INFO:teuthology.orchestra.run.smithi092.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.2 MB/s | 249 kB 00:00 2024-04-25T20:46:25.690 INFO:teuthology.orchestra.run.smithi092.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 456 kB/s | 49 kB 00:00 2024-04-25T20:46:25.774 INFO:teuthology.orchestra.run.smithi092.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 198 kB/s | 16 kB 00:00 2024-04-25T20:46:25.780 INFO:teuthology.orchestra.run.smithi047.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 8.0 MB/s | 3.5 MB 00:00 2024-04-25T20:46:25.808 INFO:teuthology.orchestra.run.smithi186.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 731 kB/s | 548 kB 00:00 2024-04-25T20:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 273 kB 00:00 2024-04-25T20:46:25.934 INFO:teuthology.orchestra.run.smithi186.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-04-25T20:46:26.034 INFO:teuthology.orchestra.run.smithi186.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 323 kB/s | 32 kB 00:00 2024-04-25T20:46:26.134 INFO:teuthology.orchestra.run.smithi186.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 761 kB/s | 838 kB 00:01 2024-04-25T20:46:26.158 INFO:teuthology.orchestra.run.smithi092.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.6 MB/s | 762 kB 00:00 2024-04-25T20:46:26.159 INFO:teuthology.orchestra.run.smithi186.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-04-25T20:46:26.250 INFO:teuthology.orchestra.run.smithi092.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 199 kB/s | 18 kB 00:00 2024-04-25T20:46:26.359 INFO:teuthology.orchestra.run.smithi092.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T20:46:26.410 INFO:teuthology.orchestra.run.smithi186.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 624 kB/s | 172 kB 00:00 2024-04-25T20:46:26.443 INFO:teuthology.orchestra.run.smithi092.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-04-25T20:46:26.476 INFO:teuthology.orchestra.run.smithi092.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 5.8 MB/s | 195 kB 00:00 2024-04-25T20:46:26.518 INFO:teuthology.orchestra.run.smithi092.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T20:46:26.555 INFO:teuthology.orchestra.run.smithi047.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-04-25T20:46:26.585 INFO:teuthology.orchestra.run.smithi092.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T20:46:26.619 INFO:teuthology.orchestra.run.smithi092.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T20:46:26.647 INFO:teuthology.orchestra.run.smithi047.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.7 MB/s | 249 kB 00:00 2024-04-25T20:46:26.661 INFO:teuthology.orchestra.run.smithi092.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T20:46:26.668 INFO:teuthology.orchestra.run.smithi186.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-04-25T20:46:26.703 INFO:teuthology.orchestra.run.smithi092.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T20:46:26.722 INFO:teuthology.orchestra.run.smithi047.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 657 kB/s | 49 kB 00:00 2024-04-25T20:46:26.744 INFO:teuthology.orchestra.run.smithi092.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T20:46:26.785 INFO:teuthology.orchestra.run.smithi186.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 303 kB/s | 35 kB 00:00 2024-04-25T20:46:26.789 INFO:teuthology.orchestra.run.smithi047.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 247 kB/s | 16 kB 00:00 2024-04-25T20:46:26.794 INFO:teuthology.orchestra.run.smithi092.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T20:46:26.860 INFO:teuthology.orchestra.run.smithi186.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 673 kB/s | 303 kB 00:00 2024-04-25T20:46:26.862 INFO:teuthology.orchestra.run.smithi092.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T20:46:26.904 INFO:teuthology.orchestra.run.smithi092.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-25T20:46:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 648 kB/s | 21 kB 00:00 2024-04-25T20:46:26.987 INFO:teuthology.orchestra.run.smithi092.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T20:46:27.006 INFO:teuthology.orchestra.run.smithi047.stdout:(35/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.0 MB/s | 21 MB 00:10 2024-04-25T20:46:27.021 INFO:teuthology.orchestra.run.smithi092.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 980 kB/s | 33 kB 00:00 2024-04-25T20:46:27.031 INFO:teuthology.orchestra.run.smithi047.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.1 MB/s | 273 kB 00:00 2024-04-25T20:46:27.107 INFO:teuthology.orchestra.run.smithi047.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 243 kB/s | 18 kB 00:00 2024-04-25T20:46:27.127 INFO:teuthology.orchestra.run.smithi186.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 601 kB/s | 206 kB 00:00 2024-04-25T20:46:27.130 INFO:teuthology.orchestra.run.smithi092.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T20:46:27.164 INFO:teuthology.orchestra.run.smithi092.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-04-25T20:46:27.197 INFO:teuthology.orchestra.run.smithi092.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-04-25T20:46:27.207 INFO:teuthology.orchestra.run.smithi047.stdout:(38/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.7 MB/s | 762 kB 00:00 2024-04-25T20:46:27.232 INFO:teuthology.orchestra.run.smithi047.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-04-25T20:46:27.239 INFO:teuthology.orchestra.run.smithi092.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T20:46:27.273 INFO:teuthology.orchestra.run.smithi092.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T20:46:27.291 INFO:teuthology.orchestra.run.smithi047.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-04-25T20:46:27.316 INFO:teuthology.orchestra.run.smithi047.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T20:46:27.323 INFO:teuthology.orchestra.run.smithi092.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T20:46:27.342 INFO:teuthology.orchestra.run.smithi047.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T20:46:27.383 INFO:teuthology.orchestra.run.smithi047.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T20:46:27.390 INFO:teuthology.orchestra.run.smithi092.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-25T20:46:27.394 INFO:teuthology.orchestra.run.smithi186.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 542 kB/s | 144 kB 00:00 2024-04-25T20:46:27.408 INFO:teuthology.orchestra.run.smithi047.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T20:46:27.434 INFO:teuthology.orchestra.run.smithi047.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T20:46:27.460 INFO:teuthology.orchestra.run.smithi047.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T20:46:27.474 INFO:teuthology.orchestra.run.smithi092.stdout:(60/152): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-04-25T20:46:27.485 INFO:teuthology.orchestra.run.smithi047.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T20:46:27.499 INFO:teuthology.orchestra.run.smithi092.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-04-25T20:46:27.519 INFO:teuthology.orchestra.run.smithi186.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-04-25T20:46:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 909 kB/s | 47 kB 00:00 2024-04-25T20:46:27.552 INFO:teuthology.orchestra.run.smithi092.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 292 kB 00:00 2024-04-25T20:46:27.569 INFO:teuthology.orchestra.run.smithi047.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-04-25T20:46:27.586 INFO:teuthology.orchestra.run.smithi092.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T20:46:27.610 INFO:teuthology.orchestra.run.smithi047.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-04-25T20:46:27.611 INFO:teuthology.orchestra.run.smithi092.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-25T20:46:27.636 INFO:teuthology.orchestra.run.smithi047.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T20:46:27.636 INFO:teuthology.orchestra.run.smithi186.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-25T20:46:27.661 INFO:teuthology.orchestra.run.smithi047.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T20:46:27.694 INFO:teuthology.orchestra.run.smithi047.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T20:46:27.719 INFO:teuthology.orchestra.run.smithi047.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-04-25T20:46:27.745 INFO:teuthology.orchestra.run.smithi092.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-25T20:46:27.745 INFO:teuthology.orchestra.run.smithi186.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-25T20:46:27.803 INFO:teuthology.orchestra.run.smithi092.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-04-25T20:46:27.845 INFO:teuthology.orchestra.run.smithi092.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T20:46:27.870 INFO:teuthology.orchestra.run.smithi186.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 156 kB/s | 19 kB 00:00 2024-04-25T20:46:27.887 INFO:teuthology.orchestra.run.smithi092.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T20:46:27.912 INFO:teuthology.orchestra.run.smithi092.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T20:46:27.996 INFO:teuthology.orchestra.run.smithi186.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-04-25T20:46:27.996 INFO:teuthology.orchestra.run.smithi092.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-25T20:46:28.071 INFO:teuthology.orchestra.run.smithi092.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-04-25T20:46:28.096 INFO:teuthology.orchestra.run.smithi092.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 893 kB/s | 87 kB 00:00 2024-04-25T20:46:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-25T20:46:28.145 INFO:teuthology.orchestra.run.smithi047.stdout:(54/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 8.8 MB/s | 49 MB 00:05 2024-04-25T20:46:28.170 INFO:teuthology.orchestra.run.smithi047.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 40 kB/s | 18 kB 00:00 2024-04-25T20:46:28.220 INFO:teuthology.orchestra.run.smithi047.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-04-25T20:46:28.245 INFO:teuthology.orchestra.run.smithi047.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 112 kB/s | 11 kB 00:00 2024-04-25T20:46:28.271 INFO:teuthology.orchestra.run.smithi047.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-25T20:46:28.288 INFO:teuthology.orchestra.run.smithi047.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T20:46:28.313 INFO:teuthology.orchestra.run.smithi047.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-25T20:46:28.338 INFO:teuthology.orchestra.run.smithi047.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-25T20:46:28.355 INFO:teuthology.orchestra.run.smithi047.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 624 kB/s | 25 kB 00:00 2024-04-25T20:46:28.372 INFO:teuthology.orchestra.run.smithi047.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 826 kB/s | 47 kB 00:00 2024-04-25T20:46:28.397 INFO:teuthology.orchestra.run.smithi047.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T20:46:28.414 INFO:teuthology.orchestra.run.smithi047.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 857 kB/s | 35 kB 00:00 2024-04-25T20:46:28.439 INFO:teuthology.orchestra.run.smithi047.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-25T20:46:28.540 INFO:teuthology.orchestra.run.smithi047.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 36 MB/s | 4.4 MB 00:00 2024-04-25T20:46:28.555 INFO:teuthology.orchestra.run.smithi092.stdout:(75/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 5.0 MB/s | 49 MB 00:09 2024-04-25T20:46:28.580 INFO:teuthology.orchestra.run.smithi092.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 564 kB/s | 248 kB 00:00 2024-04-25T20:46:28.590 INFO:teuthology.orchestra.run.smithi047.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T20:46:28.657 INFO:teuthology.orchestra.run.smithi047.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-25T20:46:28.664 INFO:teuthology.orchestra.run.smithi092.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-25T20:46:28.681 INFO:teuthology.orchestra.run.smithi092.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 206 kB/s | 26 kB 00:00 2024-04-25T20:46:28.682 INFO:teuthology.orchestra.run.smithi047.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 6.4 MB/s | 911 kB 00:00 2024-04-25T20:46:28.698 INFO:teuthology.orchestra.run.smithi092.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 167 kB/s | 19 kB 00:00 2024-04-25T20:46:28.699 INFO:teuthology.orchestra.run.smithi047.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T20:46:28.715 INFO:teuthology.orchestra.run.smithi092.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-04-25T20:46:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T20:46:28.741 INFO:teuthology.orchestra.run.smithi047.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-04-25T20:46:28.765 INFO:teuthology.orchestra.run.smithi092.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-04-25T20:46:28.774 INFO:teuthology.orchestra.run.smithi047.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-25T20:46:28.799 INFO:teuthology.orchestra.run.smithi092.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-25T20:46:28.824 INFO:teuthology.orchestra.run.smithi092.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 77 kB 00:00 2024-04-25T20:46:28.892 INFO:teuthology.orchestra.run.smithi092.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-04-25T20:46:28.917 INFO:teuthology.orchestra.run.smithi047.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-25T20:46:28.917 INFO:teuthology.orchestra.run.smithi092.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 369 kB/s | 42 kB 00:00 2024-04-25T20:46:28.942 INFO:teuthology.orchestra.run.smithi047.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T20:46:28.950 INFO:teuthology.orchestra.run.smithi092.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-25T20:46:28.984 INFO:teuthology.orchestra.run.smithi092.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-25T20:46:29.009 INFO:teuthology.orchestra.run.smithi092.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 877 kB/s | 51 kB 00:00 2024-04-25T20:46:29.034 INFO:teuthology.orchestra.run.smithi047.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-04-25T20:46:29.034 INFO:teuthology.orchestra.run.smithi092.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-25T20:46:29.059 INFO:teuthology.orchestra.run.smithi047.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-25T20:46:29.059 INFO:teuthology.orchestra.run.smithi092.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 715 kB/s | 54 kB 00:00 2024-04-25T20:46:29.084 INFO:teuthology.orchestra.run.smithi047.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 184 kB/s | 26 kB 00:00 2024-04-25T20:46:29.102 INFO:teuthology.orchestra.run.smithi092.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-25T20:46:29.109 INFO:teuthology.orchestra.run.smithi047.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T20:46:29.127 INFO:teuthology.orchestra.run.smithi092.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 689 kB/s | 44 kB 00:00 2024-04-25T20:46:29.134 INFO:teuthology.orchestra.run.smithi047.stdout:(80/152): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-04-25T20:46:29.160 INFO:teuthology.orchestra.run.smithi047.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-04-25T20:46:29.169 INFO:teuthology.orchestra.run.smithi092.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-25T20:46:29.194 INFO:teuthology.orchestra.run.smithi092.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 562 kB/s | 36 kB 00:00 2024-04-25T20:46:29.235 INFO:teuthology.orchestra.run.smithi047.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.0 MB/s | 222 kB 00:00 2024-04-25T20:46:29.236 INFO:teuthology.orchestra.run.smithi092.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 748 kB/s | 47 kB 00:00 2024-04-25T20:46:29.261 INFO:teuthology.orchestra.run.smithi047.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-04-25T20:46:29.270 INFO:teuthology.orchestra.run.smithi092.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.1 MB/s | 305 kB 00:00 2024-04-25T20:46:29.436 INFO:teuthology.orchestra.run.smithi047.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-25T20:46:29.436 INFO:teuthology.orchestra.run.smithi092.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-25T20:46:29.452 INFO:teuthology.orchestra.run.smithi047.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 356 kB/s | 77 kB 00:00 2024-04-25T20:46:29.462 INFO:teuthology.orchestra.run.smithi092.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 259 kB/s | 58 kB 00:00 2024-04-25T20:46:29.478 INFO:teuthology.orchestra.run.smithi047.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 197 kB/s | 42 kB 00:00 2024-04-25T20:46:29.479 INFO:teuthology.orchestra.run.smithi186.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 705 kB/s | 1.0 MB 00:01 2024-04-25T20:46:29.495 INFO:teuthology.orchestra.run.smithi092.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 420 kB/s | 94 kB 00:00 2024-04-25T20:46:29.503 INFO:teuthology.orchestra.run.smithi047.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T20:46:29.520 INFO:teuthology.orchestra.run.smithi092.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T20:46:29.528 INFO:teuthology.orchestra.run.smithi047.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T20:46:29.537 INFO:teuthology.orchestra.run.smithi186.stdout:(129/152): libarrow-9.0.0-10.el9.x86_64.rpm 799 kB/s | 4.4 MB 00:05 2024-04-25T20:46:29.545 INFO:teuthology.orchestra.run.smithi092.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 539 kB/s | 45 kB 00:00 2024-04-25T20:46:29.553 INFO:teuthology.orchestra.run.smithi047.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T20:46:29.571 INFO:teuthology.orchestra.run.smithi092.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-25T20:46:29.578 INFO:teuthology.orchestra.run.smithi047.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 687 kB/s | 51 kB 00:00 2024-04-25T20:46:29.579 INFO:teuthology.orchestra.run.smithi186.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 767 kB/s | 2.0 MB 00:02 2024-04-25T20:46:29.604 INFO:teuthology.orchestra.run.smithi047.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T20:46:29.605 INFO:teuthology.orchestra.run.smithi186.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-04-25T20:46:29.621 INFO:teuthology.orchestra.run.smithi092.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T20:46:29.629 INFO:teuthology.orchestra.run.smithi047.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-04-25T20:46:29.663 INFO:teuthology.orchestra.run.smithi047.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 774 kB/s | 44 kB 00:00 2024-04-25T20:46:29.696 INFO:teuthology.orchestra.run.smithi047.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.4 MB/s | 221 kB 00:00 2024-04-25T20:46:29.722 INFO:teuthology.orchestra.run.smithi047.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 648 kB/s | 36 kB 00:00 2024-04-25T20:46:29.738 INFO:teuthology.orchestra.run.smithi186.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 393 kB/s | 79 kB 00:00 2024-04-25T20:46:29.755 INFO:teuthology.orchestra.run.smithi047.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 853 kB/s | 47 kB 00:00 2024-04-25T20:46:29.781 INFO:teuthology.orchestra.run.smithi047.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.4 MB/s | 305 kB 00:00 2024-04-25T20:46:29.806 INFO:teuthology.orchestra.run.smithi047.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T20:46:29.831 INFO:teuthology.orchestra.run.smithi047.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 94 kB 00:00 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi186.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 203 kB/s | 58 kB 00:00 2024-04-25T20:46:29.881 INFO:teuthology.orchestra.run.smithi186.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 115 kB/s | 16 kB 00:00 2024-04-25T20:46:29.990 INFO:teuthology.orchestra.run.smithi047.stdout:(100/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-25T20:46:30.006 INFO:teuthology.orchestra.run.smithi186.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-04-25T20:46:30.015 INFO:teuthology.orchestra.run.smithi047.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 310 kB/s | 64 kB 00:00 2024-04-25T20:46:30.030 INFO:teuthology.orchestra.run.smithi092.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 48 kB/s | 19 kB 00:00 2024-04-25T20:46:30.040 INFO:teuthology.orchestra.run.smithi047.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 217 kB/s | 45 kB 00:00 2024-04-25T20:46:30.048 INFO:teuthology.orchestra.run.smithi186.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 489 kB/s | 90 kB 00:00 2024-04-25T20:46:30.065 INFO:teuthology.orchestra.run.smithi047.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-25T20:46:30.082 INFO:teuthology.orchestra.run.smithi186.stdout:(137/152): python3-pecan-1.4.2-3.el9.noarch.rpm 572 kB/s | 272 kB 00:00 2024-04-25T20:46:30.091 INFO:teuthology.orchestra.run.smithi047.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T20:46:30.107 INFO:teuthology.orchestra.run.smithi186.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-25T20:46:30.205 INFO:teuthology.orchestra.run.smithi092.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 488 kB/s | 309 kB 00:00 2024-04-25T20:46:30.215 INFO:teuthology.orchestra.run.smithi186.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-25T20:46:30.257 INFO:teuthology.orchestra.run.smithi186.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 337 kB/s | 59 kB 00:00 2024-04-25T20:46:30.297 INFO:teuthology.orchestra.run.smithi092.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 730 kB/s | 548 kB 00:00 2024-04-25T20:46:30.322 INFO:teuthology.orchestra.run.smithi092.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-04-25T20:46:30.324 INFO:teuthology.orchestra.run.smithi186.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 683 kB/s | 188 kB 00:00 2024-04-25T20:46:30.341 INFO:teuthology.orchestra.run.smithi186.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-04-25T20:46:30.391 INFO:teuthology.orchestra.run.smithi047.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-04-25T20:46:30.406 INFO:teuthology.orchestra.run.smithi092.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-25T20:46:30.439 INFO:teuthology.orchestra.run.smithi092.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-04-25T20:46:30.517 INFO:teuthology.orchestra.run.smithi186.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 468 kB/s | 90 kB 00:00 2024-04-25T20:46:30.523 INFO:teuthology.orchestra.run.smithi092.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T20:46:30.608 INFO:teuthology.orchestra.run.smithi047.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-04-25T20:46:30.616 INFO:teuthology.orchestra.run.smithi186.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 640 kB/s | 230 kB 00:00 2024-04-25T20:46:30.633 INFO:teuthology.orchestra.run.smithi186.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-25T20:46:30.650 INFO:teuthology.orchestra.run.smithi047.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 900 kB/s | 548 kB 00:00 2024-04-25T20:46:30.682 INFO:teuthology.orchestra.run.smithi092.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-25T20:46:30.716 INFO:teuthology.orchestra.run.smithi092.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 838 kB 00:00 2024-04-25T20:46:30.791 INFO:teuthology.orchestra.run.smithi092.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-04-25T20:46:30.816 INFO:teuthology.orchestra.run.smithi092.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 325 kB/s | 43 kB 00:00 2024-04-25T20:46:30.841 INFO:teuthology.orchestra.run.smithi092.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-25T20:46:30.867 INFO:teuthology.orchestra.run.smithi047.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-04-25T20:46:30.884 INFO:teuthology.orchestra.run.smithi186.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 718 kB/s | 192 kB 00:00 2024-04-25T20:46:30.893 INFO:teuthology.orchestra.run.smithi092.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-25T20:46:30.959 INFO:teuthology.orchestra.run.smithi092.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T20:46:30.984 INFO:teuthology.orchestra.run.smithi092.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-25T20:46:30.992 INFO:teuthology.orchestra.run.smithi186.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 656 kB/s | 427 kB 00:00 2024-04-25T20:46:31.000 INFO:teuthology.orchestra.run.smithi047.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-04-25T20:46:31.009 INFO:teuthology.orchestra.run.smithi092.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T20:46:31.059 INFO:teuthology.orchestra.run.smithi092.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-25T20:46:31.093 INFO:teuthology.orchestra.run.smithi092.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-04-25T20:46:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T20:46:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-25T20:46:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-25T20:46:31.225 INFO:teuthology.orchestra.run.smithi047.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 188 kB/s | 67 kB 00:00 2024-04-25T20:46:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T20:46:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-25T20:46:31.305 INFO:teuthology.orchestra.run.smithi092.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-25T20:46:31.349 INFO:teuthology.orchestra.run.smithi186.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 8.8 MB/s | 3.1 MB 00:00 2024-04-25T20:46:31.353 INFO:teuthology.orchestra.run.smithi092.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-25T20:46:31.401 INFO:teuthology.orchestra.run.smithi186.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 26 MB/s | 1.3 MB 00:00 2024-04-25T20:46:31.403 INFO:teuthology.orchestra.run.smithi092.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-25T20:46:31.426 INFO:teuthology.orchestra.run.smithi047.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-25T20:46:31.454 INFO:teuthology.orchestra.run.smithi092.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2024-04-25T20:46:31.487 INFO:teuthology.orchestra.run.smithi092.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-25T20:46:31.493 INFO:teuthology.orchestra.run.smithi186.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-25T20:46:31.512 INFO:teuthology.orchestra.run.smithi092.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-04-25T20:46:31.526 INFO:teuthology.orchestra.run.smithi047.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-04-25T20:46:31.563 INFO:teuthology.orchestra.run.smithi092.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T20:46:31.588 INFO:teuthology.orchestra.run.smithi092.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-25T20:46:31.613 INFO:teuthology.orchestra.run.smithi092.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-04-25T20:46:31.664 INFO:teuthology.orchestra.run.smithi092.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-25T20:46:31.689 INFO:teuthology.orchestra.run.smithi092.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-25T20:46:31.722 INFO:teuthology.orchestra.run.smithi092.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T20:46:31.756 INFO:teuthology.orchestra.run.smithi092.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-04-25T20:46:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-04-25T20:46:31.801 INFO:teuthology.orchestra.run.smithi186.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.7 MB/s | 3.4 MB 00:00 2024-04-25T20:46:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-25T20:46:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T20:46:31.868 INFO:teuthology.orchestra.run.smithi047.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-25T20:46:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 891 kB/s | 90 kB 00:00 2024-04-25T20:46:31.901 INFO:teuthology.orchestra.run.smithi047.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-25T20:46:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T20:46:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-25T20:46:31.991 INFO:teuthology.orchestra.run.smithi092.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T20:46:32.052 INFO:teuthology.orchestra.run.smithi047.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 876 kB/s | 548 kB 00:00 2024-04-25T20:46:32.135 INFO:teuthology.orchestra.run.smithi047.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 120 kB/s | 32 kB 00:00 2024-04-25T20:46:32.158 INFO:teuthology.orchestra.run.smithi092.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-25T20:46:32.160 INFO:teuthology.orchestra.run.smithi047.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 58 kB/s | 15 kB 00:00 2024-04-25T20:46:32.233 INFO:teuthology.orchestra.run.smithi092.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T20:46:32.334 INFO:teuthology.orchestra.run.smithi092.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-25T20:46:32.451 INFO:teuthology.orchestra.run.smithi092.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 6.8 MB/s | 3.1 MB 00:00 2024-04-25T20:46:32.486 INFO:teuthology.orchestra.run.smithi047.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 396 kB/s | 172 kB 00:00 2024-04-25T20:46:32.709 INFO:teuthology.orchestra.run.smithi092.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 4.6 MB/s | 3.4 MB 00:00 2024-04-25T20:46:32.710 INFO:teuthology.orchestra.run.smithi186.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 780 kB/s | 1.6 MB 00:02 2024-04-25T20:46:32.716 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:46:32.716 INFO:teuthology.orchestra.run.smithi092.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-25T20:46:32.718 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:46:32.718 INFO:teuthology.orchestra.run.smithi186.stdout:Total 11 MB/s | 222 MB 00:20 2024-04-25T20:46:32.820 INFO:teuthology.orchestra.run.smithi047.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 524 kB/s | 358 kB 00:00 2024-04-25T20:46:32.845 INFO:teuthology.orchestra.run.smithi047.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 98 kB/s | 35 kB 00:00 2024-04-25T20:46:32.870 INFO:teuthology.orchestra.run.smithi047.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 427 kB/s | 303 kB 00:00 2024-04-25T20:46:33.312 INFO:teuthology.orchestra.run.smithi047.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 327 kB/s | 144 kB 00:00 2024-04-25T20:46:33.337 INFO:teuthology.orchestra.run.smithi047.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 397 kB/s | 206 kB 00:00 2024-04-25T20:46:33.496 INFO:teuthology.orchestra.run.smithi047.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-04-25T20:46:33.613 INFO:teuthology.orchestra.run.smithi047.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 64 kB/s | 18 kB 00:00 2024-04-25T20:46:33.655 INFO:teuthology.orchestra.run.smithi047.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-04-25T20:46:33.766 INFO:teuthology.orchestra.run.smithi047.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 86 kB/s | 23 kB 00:00 2024-04-25T20:46:33.813 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-25T20:46:33.847 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-25T20:46:33.880 INFO:teuthology.orchestra.run.smithi047.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-25T20:46:33.947 INFO:teuthology.orchestra.run.smithi047.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 67 kB/s | 20 kB 00:00 2024-04-25T20:46:33.957 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-25T20:46:33.957 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-25T20:46:33.993 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-25T20:46:33.993 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-25T20:46:34.223 INFO:teuthology.orchestra.run.smithi047.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-25T20:46:34.290 INFO:teuthology.orchestra.run.smithi047.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 230 kB/s | 79 kB 00:00 2024-04-25T20:46:34.440 INFO:teuthology.orchestra.run.smithi047.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-25T20:46:34.565 INFO:teuthology.orchestra.run.smithi047.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 169 kB/s | 58 kB 00:00 2024-04-25T20:46:34.707 INFO:teuthology.orchestra.run.smithi047.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-25T20:46:34.791 INFO:teuthology.orchestra.run.smithi047.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-04-25T20:46:34.907 INFO:teuthology.orchestra.run.smithi047.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 263 kB/s | 90 kB 00:00 2024-04-25T20:46:34.983 INFO:teuthology.orchestra.run.smithi047.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 102 kB/s | 28 kB 00:00 2024-04-25T20:46:35.058 INFO:teuthology.orchestra.run.smithi047.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-25T20:46:35.333 INFO:teuthology.orchestra.run.smithi047.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-04-25T20:46:35.358 INFO:teuthology.orchestra.run.smithi047.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 119 kB/s | 36 kB 00:00 2024-04-25T20:46:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 396 kB/s | 188 kB 00:00 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi047.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 142 kB/s | 51 kB 00:00 2024-04-25T20:46:35.717 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-25T20:46:35.717 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-25T20:46:35.743 INFO:teuthology.orchestra.run.smithi047.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 250 kB/s | 90 kB 00:00 2024-04-25T20:46:35.779 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-25T20:46:35.780 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-25T20:46:35.868 INFO:teuthology.orchestra.run.smithi047.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-04-25T20:46:36.001 INFO:teuthology.orchestra.run.smithi047.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 77 kB/s | 20 kB 00:00 2024-04-25T20:46:36.277 INFO:teuthology.orchestra.run.smithi047.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-25T20:46:36.302 INFO:teuthology.orchestra.run.smithi047.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 442 kB/s | 192 kB 00:00 2024-04-25T20:46:36.727 INFO:teuthology.orchestra.run.smithi047.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 7.5 MB/s | 3.4 MB 00:00 2024-04-25T20:46:36.769 INFO:teuthology.orchestra.run.smithi047.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-25T20:46:36.802 INFO:teuthology.orchestra.run.smithi047.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T20:46:36.861 INFO:teuthology.orchestra.run.smithi047.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-25T20:46:36.911 INFO:teuthology.orchestra.run.smithi047.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-04-25T20:46:36.916 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:46:36.917 INFO:teuthology.orchestra.run.smithi047.stdout:Total 11 MB/s | 222 MB 00:20 2024-04-25T20:46:38.056 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-25T20:46:38.215 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-25T20:46:38.215 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-25T20:46:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-25T20:46:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-25T20:46:41.767 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-25T20:46:41.847 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-25T20:46:41.872 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:46:41.919 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:46:41.937 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:46:42.065 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:46:42.068 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:46:42.233 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:46:42.257 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:46:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:46:42.302 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:46:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:46:42.347 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:46:42.392 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:46:42.629 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:46:42.749 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:46:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:46:42.871 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:46:42.900 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:46:42.964 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:46:42.991 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:46:43.030 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:46:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:46:43.116 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:46:43.179 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:46:43.206 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:46:43.226 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:46:43.237 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:46:43.255 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:46:43.359 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:46:43.380 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:46:43.391 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:46:43.426 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:46:43.442 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:46:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:46:43.476 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:46:43.497 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:46:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:46:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:46:43.673 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:46:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:46:43.705 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:46:43.706 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:46:43.817 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:46:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:46:43.919 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:46:43.943 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:46:43.956 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:46:43.991 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:46:44.064 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:46:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:46:44.123 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:46:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:46:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:46:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:46:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:46:44.336 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:46:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:46:44.374 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:46:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:46:44.402 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:46:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:46:44.460 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:46:44.471 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:46:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:46:44.532 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:46:44.563 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:46:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:46:44.598 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:46:44.675 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:46:44.709 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:46:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:46:44.753 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:46:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:46:44.772 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:46:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:46:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:46:44.843 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:46:44.887 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:46:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:46:44.962 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:46:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:46:45.093 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:46:45.147 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:46:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:46:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:46:45.661 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:46:45.743 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:46:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:46:45.772 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:46:45.818 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:46:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:46:45.861 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:46:45.903 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:46:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:46:46.052 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:46:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-25T20:46:46.114 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:46:46.169 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:46:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:46:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:46:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:46:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:46:46.241 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:46:46.273 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:46:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:46:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:46:46.309 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:46:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:46:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:46:46.355 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:46:46.366 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:46:46.392 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:46:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:46:46.397 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:46:46.435 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:46:46.442 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:46:46.477 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:46:46.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:46:46.533 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:46:46.576 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:46:46.614 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:46:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:46:46.666 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:46:46.696 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:46:46.737 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:46:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:46:46.755 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:46:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:46:46.832 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:46:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:46:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:46:46.871 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:46:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:46:46.911 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:46:46.936 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:46:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:46:46.961 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:46:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:46:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:46:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:46:47.036 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:46:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:46:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:46:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:46:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:46:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:46:47.162 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:46:47.194 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:46:47.201 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:46:47.229 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:46:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:46:47.264 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:46:47.303 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:46:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:46:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:46:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:46:47.334 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:46:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:46:47.389 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:46:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:46:47.423 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:46:47.435 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:46:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:46:47.457 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:46:47.465 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:46:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:46:47.498 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:46:47.518 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:46:47.519 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:46:47.541 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:46:47.541 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:46:47.548 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:46:47.569 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:46:47.583 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:46:47.616 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:46:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:46:47.658 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:46:47.671 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:46:47.692 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:46:47.703 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:46:47.750 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:46:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:46:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:46:47.803 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:46:47.848 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:46:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:46:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:46:47.896 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:46:47.909 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:46:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:46:48.005 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:46:48.032 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:46:48.039 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:46:48.080 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:46:48.083 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:46:48.161 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:46:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:46:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:46:48.275 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:46:48.303 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:46:48.366 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:46:48.417 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:46:48.447 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:46:48.491 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:46:48.569 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:46:48.601 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:46:48.619 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:46:48.654 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:46:48.660 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:46:48.682 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:46:48.693 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:46:48.768 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:46:48.800 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:46:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:46:48.835 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:46:48.852 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:46:48.852 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:46:48.894 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:46:48.930 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:46:48.937 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:46:48.996 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:46:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:46:49.124 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:46:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:46:49.173 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:46:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:46:49.201 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:46:49.288 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:46:49.372 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:46:49.411 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:46:49.488 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:46:49.677 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:46:49.763 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:46:49.798 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:46:49.841 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:46:49.879 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:46:49.921 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:46:49.933 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:46:50.057 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:46:50.085 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:46:50.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:46:50.149 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:46:50.205 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:46:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:46:50.279 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:46:50.315 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:46:50.352 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:46:50.400 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:46:50.435 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:46:50.478 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:46:50.520 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:46:50.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:46:50.709 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:46:50.739 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:46:50.781 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:46:50.880 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:46:50.913 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:46:50.954 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:46:50.978 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:46:51.003 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:46:51.081 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:46:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:46:51.261 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:46:51.279 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:46:51.289 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:46:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:46:51.363 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:46:51.371 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:46:51.382 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:46:51.407 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:46:51.411 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:46:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:46:51.475 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:46:51.494 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:46:51.508 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:46:51.551 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:46:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:46:51.587 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:46:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:46:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:46:51.674 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:46:51.719 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:46:51.723 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:46:51.757 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:46:51.762 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:46:51.802 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:46:51.853 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:46:51.883 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:46:51.914 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:46:51.936 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:46:51.944 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:46:51.973 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:46:51.988 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:46:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:46:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:46:52.145 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:46:52.414 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:46:52.439 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:46:52.511 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:46:52.541 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:46:52.632 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:46:52.666 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:46:52.737 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:46:52.742 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:46:52.768 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:46:52.785 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:46:52.819 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:46:52.922 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:46:52.978 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:46:53.067 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:46:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:46:53.288 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:46:53.318 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:46:53.397 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:46:54.075 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:46:54.221 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:46:54.820 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:54.893 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:55.017 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:55.064 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:55.442 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:55.483 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:46:55.514 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:46:55.581 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:46:55.581 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:46:55.581 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:46:55.581 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:46:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:46:55.608 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:46:55.625 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:46:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:46:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:46:55.733 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:46:55.733 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:46:55.733 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:46:55.733 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:46:55.768 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:46:55.977 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:46:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:46:56.120 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:46:56.150 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:46:56.187 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:46:56.694 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:46:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:46:56.778 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:46:56.808 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:46:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:46:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:46:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:46:59.917 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:46:59.917 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:46:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:46:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /root 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-04-25T20:47:06.941 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:07.120 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:47:07.137 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:47:07.137 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:07.137 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:47:07.137 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:07.309 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:47:07.309 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-04-25T20:47:07.309 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-04-25T20:47:07.310 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-04-25T20:47:07.310 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-04-25T20:47:07.310 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-04-25T20:47:07.310 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-04-25T20:47:07.310 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-04-25T20:47:07.310 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:07.479 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:47:07.494 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:47:07.494 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:07.495 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:47:07.495 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:08.623 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:47:08.649 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:47:08.649 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:08.649 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:47:08.649 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:47:08.649 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:47:08.649 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:08.873 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:47:08.896 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:47:08.896 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:08.896 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:47:08.896 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:47:08.896 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:47:08.896 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:09.216 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:47:09.238 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:47:09.238 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:09.238 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:47:09.238 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:47:09.239 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:47:09.239 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:09.340 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:47:09.363 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:47:09.364 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:09.364 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:47:09.364 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:47:09.364 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:47:09.364 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:09.462 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:47:09.510 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:47:09.566 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:47:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:47:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:47:09.706 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:09.706 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:47:09.706 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:47:09.706 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:47:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:09.750 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:47:09.799 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:47:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:47:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:47:09.876 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:47:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:47:09.965 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:47:10.095 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:47:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:47:10.132 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:47:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:47:10.164 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:47:10.192 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:47:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:47:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:47:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:47:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:47:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:47:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:47:10.594 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:47:10.624 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:47:10.665 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:47:10.725 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:47:10.763 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:47:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:47:11.017 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:47:11.049 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:47:11.111 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:47:11.149 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-04-25T20:47:11.705 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:11.802 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:47:11.832 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:47:11.896 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:47:11.912 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:47:11.912 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:11.913 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:47:11.913 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:47:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:47:12.798 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:47:12.832 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:47:12.936 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:47:13.096 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:47:13.126 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:47:13.149 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:47:13.149 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:13.150 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:47:13.150 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:47:13.150 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:47:13.150 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:47:13.193 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:47:13.204 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:47:13.225 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:47:13.302 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:47:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:47:13.358 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:47:13.449 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:47:13.476 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:47:13.476 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:13.476 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:47:13.477 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:47:13.477 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:47:13.477 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:13.488 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:47:13.524 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:47:13.547 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:47:13.547 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:13.548 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:47:13.548 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:47:13.548 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:47:13.548 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:13.595 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:47:13.606 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:47:13.703 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:47:13.708 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:47:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:47:13.731 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:13.732 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:47:13.732 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:47:13.732 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:47:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:13.751 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:47:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:47:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:47:14.216 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:14.216 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:47:14.216 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:47:14.216 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:47:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:14.284 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:47:14.313 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:47:14.322 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:47:14.336 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:47:14.336 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:14.336 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:47:14.336 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:47:14.336 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:47:14.336 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:14.380 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:47:14.380 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:47:14.402 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:47:14.402 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:14.402 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:47:14.402 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:14.447 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:47:14.452 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:47:14.469 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:47:14.469 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:14.469 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:47:14.469 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:47:14.470 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:47:14.470 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:14.648 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:47:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:47:14.693 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:47:14.723 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:47:14.745 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:47:14.746 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:14.746 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:47:14.746 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:47:14.746 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:47:14.746 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:14.773 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:47:14.781 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:47:14.790 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:47:14.803 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:47:14.804 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:14.804 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:47:14.804 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:47:14.804 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:47:14.804 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:14.812 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:47:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:47:14.812 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:47:14.863 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:47:14.879 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:47:14.879 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:14.879 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:47:14.880 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:47:14.880 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:47:14.880 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:15.007 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:47:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:47:15.074 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:47:15.101 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:47:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:47:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:47:15.205 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:15.206 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:47:15.206 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:47:15.206 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:47:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:15.547 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:47:15.600 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:47:15.633 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:47:15.697 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:47:15.732 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:47:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:47:16.866 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:47:17.880 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:47:17.908 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:47:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:47:18.187 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:47:18.210 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:47:18.235 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:47:18.236 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:18.236 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:47:18.236 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:47:18.236 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:47:18.236 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:18.286 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:47:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:47:18.391 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:47:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:47:19.384 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:47:19.417 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:47:19.434 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:47:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:47:19.475 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:47:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:47:19.480 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:19.480 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:47:19.480 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:47:19.480 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:47:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:19.507 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:47:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:47:19.534 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:47:19.546 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:47:19.547 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:19.547 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:47:19.547 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:19.598 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:47:19.622 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:47:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:47:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:47:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:47:19.623 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:19.719 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:47:19.772 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:47:19.794 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:47:19.810 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:47:19.843 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:47:19.893 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:47:19.925 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:47:19.941 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:47:19.952 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:47:19.964 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:47:19.965 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:47:19.965 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:47:19.965 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:47:19.965 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:47:19.965 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:20.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:47:20.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:47:20.211 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:47:20.399 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:20.432 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:20.453 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:20.819 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:21.291 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:47:21.378 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:47:21.419 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:47:21.504 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:47:21.532 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:47:21.533 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:47:21.624 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:47:21.666 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:47:21.666 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:47:21.713 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:47:21.755 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:47:21.823 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:47:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:47:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:47:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:47:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:47:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:47:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:47:22.143 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:47:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:24.803 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:47:24.844 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:47:24.894 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:47:24.926 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:47:24.953 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:47:25.147 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:47:25.191 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:47:25.220 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:47:25.976 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:26.009 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:26.031 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:47:26.943 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:47:27.030 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:47:27.081 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:47:27.174 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:47:27.210 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:47:27.210 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:47:27.302 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:47:27.356 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:47:27.356 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:47:27.426 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:47:27.495 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:27.768 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:27.768 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:47:27.768 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:47:27.768 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:47:27.768 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:47:27.768 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:47:27.768 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:47:27.769 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:47:27.770 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:47:27.771 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:47:27.772 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:47:27.773 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:47:27.774 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:47:27.775 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:47:27.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:47:27.777 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:47:27.778 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:47:27.779 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:47:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:47:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:47:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:47:27.783 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:47:27.783 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:47:27.783 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:47:27.783 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:47:27.783 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:47:27.783 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:47:27.783 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:47:27.783 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:47:27.784 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:47:27.784 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:47:27.784 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:47:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:47:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:47:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:47:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:47:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:47:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:47:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:47:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:47:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:47:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:47:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:47:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:47:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:47:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:47:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:47:29.784 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:47:29.785 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.786 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.787 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:47:29.788 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:47:29.789 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:47:29.790 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:47:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:47:29.792 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:47:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.794 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:47:29.796 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:47:29.797 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:47:29.797 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:47:29.797 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-25T20:47:30.184 DEBUG:teuthology.parallel:result is None 2024-04-25T20:47:30.326 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.327 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.328 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:47:30.329 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:47:30.330 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:47:30.331 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:47:30.333 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:47:30.334 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.335 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:47:30.336 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:47:30.337 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:47:30.338 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:47:30.338 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-25T20:47:30.338 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-25T20:47:30.694 DEBUG:teuthology.parallel:result is None 2024-04-25T20:47:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:47:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:47:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:47:32.896 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:47:32.896 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:47:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:47:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:47:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:47:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:47:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:47:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:47:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:47:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:47:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:47:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:47:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:47:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:47:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:47:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:47:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:47:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:47:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:47:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:47:35.023 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.024 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:47:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:47:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:47:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:47:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:47:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:47:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:47:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:47:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:47:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:47:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:47:35.035 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-25T20:47:35.502 DEBUG:teuthology.parallel:result is None 2024-04-25T20:47:35.503 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:47:35.642 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:47:35.664 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:47:35.665 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:47:35.665 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:47:35.666 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:47:35.884 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:47:35.909 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:47:35.909 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:47:35.909 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:47:35.911 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:47:36.124 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:47:36.149 INFO:teuthology.orchestra.run.smithi186.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:47:36.149 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:47:36.150 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:47:36.151 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T20:47:36.152 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:47:36.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:47:36.179 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:47:36.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:47:36.208 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:47:36.209 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:47:36.239 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T20:47:36.239 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:47:36.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:47:36.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:47:36.329 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:47:36.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:47:36.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:47:36.425 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:47:36.426 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:47:36.454 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:47:36.523 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T20:47:36.523 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:47:36.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:47:36.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:47:36.612 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:47:36.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:47:36.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:47:36.704 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:47:36.704 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:47:36.733 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:47:36.800 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T20:47:36.800 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:47:36.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:47:36.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:47:36.890 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:47:36.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:47:36.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:47:36.982 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:47:36.982 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:47:37.009 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:47:37.077 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T20:47:37.178 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T20:47:37.179 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:47:37.179 INFO:tasks.cephadm:Cluster fsid is 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:47:37.179 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T20:47:37.179 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.47', 'mon.b': '172.21.15.92', 'mon.c': '172.21.15.186'} 2024-04-25T20:47:37.180 INFO:tasks.cephadm:First mon is mon.a on smithi047 2024-04-25T20:47:37.180 INFO:tasks.cephadm:First mgr is x 2024-04-25T20:47:37.180 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T20:47:37.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-04-25T20:47:37.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-04-25T20:47:37.232 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2024-04-25T20:47:37.259 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T20:47:37.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:47:37.498 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T20:47:37.745 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:47:37.746 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T20:47:37.746 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:38.251 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:47 /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:38.252 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:39.069 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:47 /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:39.069 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:39.594 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:47 /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:39.594 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:39.615 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:39.633 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:47:39.670 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T20:47:39.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:47:39.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:47:39.676 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:47:41.238 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:47:41.238 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:47:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:48:14.010 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T20:48:14.010 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:48:14.010 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:48:14.010 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-04-25T20:48:14.011 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:48:14.011 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T20:48:14.011 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T20:48:33.557 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-25T20:48:33.558 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:48:33.558 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:48:33.558 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2024-04-25T20:48:33.558 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:48:33.558 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2024-04-25T20:48:33.558 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-25T20:48:36.531 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-25T20:48:36.531 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:48:36.531 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:48:36.531 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-04-25T20:48:36.531 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:48:36.531 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-25T20:48:36.531 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-25T20:48:36.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-04-25T20:48:36.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-04-25T20:48:36.624 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2024-04-25T20:48:36.656 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-04-25T20:48:36.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-04-25T20:48:36.708 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2024-04-25T20:48:36.735 INFO:tasks.cephadm:Writing seed config... 2024-04-25T20:48:36.736 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T20:48:36.736 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T20:48:36.736 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T20:48:36.736 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T20:48:36.736 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T20:48:36.737 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T20:48:36.738 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T20:48:36.739 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T20:48:36.740 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T20:48:36.740 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T20:48:36.740 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T20:48:36.740 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T20:48:36.740 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T20:48:36.740 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T20:48:36.740 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T20:48:36.741 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:48:36.741 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T20:48:36.801 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0ca8de06-0345-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T20:48:36.801 DEBUG:teuthology.orchestra.run.smithi047:mon.a> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T20:48:36.804 DEBUG:teuthology.orchestra.run.smithi047:mgr.x> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T20:48:36.848 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T20:48:36.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.47 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:48:37.051 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:48:37.051 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '0ca8de06-0345-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.47', '--skip-admin-label'] 2024-04-25T20:48:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T20:48:37.051 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying podman|docker is present... 2024-04-25T20:48:37.119 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T20:48:37.119 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying lvm2 is present... 2024-04-25T20:48:37.119 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying time synchronization is in place... 2024-04-25T20:48:37.127 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T20:48:37.127 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T20:48:37.133 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T20:48:37.133 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-25T20:48:37.140 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-04-25T20:48:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-04-25T20:48:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-04-25T20:48:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:Repeating the final host check... 2024-04-25T20:48:37.174 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T20:48:37.174 INFO:teuthology.orchestra.run.smithi047.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T20:48:37.174 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl is present 2024-04-25T20:48:37.174 INFO:teuthology.orchestra.run.smithi047.stdout:lvcreate is present 2024-04-25T20:48:37.181 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T20:48:37.181 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T20:48:37.187 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T20:48:37.187 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-25T20:48:37.194 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-04-25T20:48:37.201 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-04-25T20:48:37.201 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-04-25T20:48:37.201 INFO:teuthology.orchestra.run.smithi047.stdout:Host looks OK 2024-04-25T20:48:37.201 INFO:teuthology.orchestra.run.smithi047.stdout:Cluster fsid: 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:37.202 INFO:teuthology.orchestra.run.smithi047.stdout:Acquiring lock 139945341344928 on /run/cephadm/0ca8de06-0345-11ef-bc93-c7b262605968.lock 2024-04-25T20:48:37.202 INFO:teuthology.orchestra.run.smithi047.stdout:Lock 139945341344928 acquired on /run/cephadm/0ca8de06-0345-11ef-bc93-c7b262605968.lock 2024-04-25T20:48:37.202 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 3300 ... 2024-04-25T20:48:37.202 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 6789 ... 2024-04-25T20:48:37.202 INFO:teuthology.orchestra.run.smithi047.stdout:Base mon IP(s) is [172.21.15.47:3300, 172.21.15.47:6789], mon addrv is [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-04-25T20:48:37.206 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.47 metric 100 2024-04-25T20:48:37.206 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.47 metric 100 2024-04-25T20:48:37.208 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T20:48:37.208 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T20:48:37.208 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T20:48:37.211 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T20:48:37.211 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T20:48:37.211 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T20:48:37.211 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T20:48:37.211 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-25T20:48:37.211 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T20:48:37.212 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-04-25T20:48:37.212 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-04-25T20:48:37.212 INFO:teuthology.orchestra.run.smithi047.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T20:48:37.212 INFO:teuthology.orchestra.run.smithi047.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T20:48:37.213 INFO:teuthology.orchestra.run.smithi047.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:48:40.635 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T20:48:40.635 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:48:40.635 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T20:48:40.635 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T20:48:40.635 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T20:48:40.635 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T20:48:40.635 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T20:48:44.526 INFO:teuthology.orchestra.run.smithi047.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T20:48:44.526 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T20:48:44.527 INFO:teuthology.orchestra.run.smithi047.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T20:48:45.432 INFO:teuthology.orchestra.run.smithi047.stdout:stat: stdout 167 167 2024-04-25T20:48:45.432 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial keys... 2024-04-25T20:48:46.305 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQCtwSpm3AxBLBAAXJUOvszTw6JAe38zCbkKfQ== 2024-04-25T20:48:47.177 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQCuwSpmbkogJhAAFUX+g7pDVDoH7OwXWWk+UA== 2024-04-25T20:48:48.103 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQCvwSpmDHuYHxAAPIYkVzPx2ttayPe2xjEg6Q== 2024-04-25T20:48:48.104 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial monmap... 2024-04-25T20:48:49.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T20:48:49.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T20:48:49.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:49.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T20:48:49.054 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool for a [v2:172.21.15.47:3300,v1:172.21.15.47:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T20:48:49.055 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-04-25T20:48:49.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: set fsid to 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:49.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T20:48:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:48:49.055 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mon... 2024-04-25T20:48:50.236 INFO:teuthology.orchestra.run.smithi047.stdout:create mon.a on 2024-04-25T20:48:50.466 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T20:48:50.638 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T20:48:50.832 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0ca8de06-0345-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-0ca8de06-0345-11ef-bc93-c7b262605968.target. 2024-04-25T20:48:50.832 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0ca8de06-0345-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-0ca8de06-0345-11ef-bc93-c7b262605968.target. 2024-04-25T20:48:51.089 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.a 2024-04-25T20:48:51.089 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T20:48:51.260 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ca8de06-0345-11ef-bc93-c7b262605968.target.wants/ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0ca8de06-0345-11ef-bc93-c7b262605968@.service. 2024-04-25T20:48:51.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:51 smithi047 podman[28802]: 2024-04-25 20:48:51.431436093 +0000 UTC m=+0.024760798 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:48:51.811 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:48:51.811 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-04-25T20:48:51.819 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:48:51.819 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-25T20:48:51.819 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-04-25T20:48:51.819 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T20:48:51.820 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon to start... 2024-04-25T20:48:51.820 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon... 2024-04-25T20:48:51.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:51 smithi047 podman[28802]: 2024-04-25 20:48:51.720137626 +0000 UTC m=+0.313462320 container init 511e5fe86af03c067d78cefdc7c5d739fda58bbeaa0096e0937bf30fa1a29e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T20:48:51.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:51 smithi047 podman[28802]: 2024-04-25 20:48:51.723512011 +0000 UTC m=+0.316836697 container start 511e5fe86af03c067d78cefdc7c5d739fda58bbeaa0096e0937bf30fa1a29e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T20:48:51.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:51 smithi047 ceph-mon[28821]: mkfs 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:51.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:51 smithi047 ceph-mon[28821]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T20:48:51.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:51 smithi047 bash[28802]: 511e5fe86af03c067d78cefdc7c5d739fda58bbeaa0096e0937bf30fa1a29e00 2024-04-25T20:48:51.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:51 smithi047 systemd[1]: Started Ceph mon.a for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:48:52.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T20:48:52.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: monmap epoch 1 2024-04-25T20:48:52.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:52.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: last_changed 2024-04-25T20:48:48.454006+0000 2024-04-25T20:48:52.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:48:52.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: min_mon_release 19 (squid) 2024-04-25T20:48:52.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: election_strategy: 1 2024-04-25T20:48:52.936 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:48:52.936 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: fsmap 2024-04-25T20:48:52.936 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T20:48:52.936 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: mgrmap e1: no daemons active 2024-04-25T20:48:52.936 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:52 smithi047 ceph-mon[28821]: from='client.? 172.21.15.47:0/1681398360' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T20:48:52.937 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout id: 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout services: 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.566413s) 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout data: 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T20:48:52.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.249+0000 7fb65810c700 1 Processor -- start 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.249+0000 7fb65810c700 1 -- start start 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.249+0000 7fb65810c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb650108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.249+0000 7fb65810c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb650109060 con 0x7fb650108670 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.250+0000 7fb655ea8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb650108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.250+0000 7fb655ea8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb650108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56186/0 (socket says 172.21.15.47:56186) 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.250+0000 7fb655ea8700 1 -- 172.21.15.47:0/2668916630 learned_addr learned my addr 172.21.15.47:0/2668916630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.250+0000 7fb655ea8700 1 -- 172.21.15.47:0/2668916630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb650109820 con 0x7fb650108670 2024-04-25T20:48:52.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.250+0000 7fb655ea8700 1 --2- 172.21.15.47:0/2668916630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb650108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6440098a0 tx=0x7fb644009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2434bbab7347aa83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.250+0000 7fb654ea6700 1 -- 172.21.15.47:0/2668916630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb644004160 con 0x7fb650108670 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.250+0000 7fb654ea6700 1 -- 172.21.15.47:0/2668916630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb644004300 con 0x7fb650108670 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.250+0000 7fb654ea6700 1 -- 172.21.15.47:0/2668916630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6440036a0 con 0x7fb650108670 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.251+0000 7fb65810c700 1 -- 172.21.15.47:0/2668916630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 msgr2=0x7fb650108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.251+0000 7fb65810c700 1 --2- 172.21.15.47:0/2668916630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb650108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6440098a0 tx=0x7fb644009bb0 comp rx=0 tx=0).stop 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.251+0000 7fb65810c700 1 -- 172.21.15.47:0/2668916630 shutdown_connections 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.251+0000 7fb65810c700 1 --2- 172.21.15.47:0/2668916630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb650108a90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.251+0000 7fb65810c700 1 -- 172.21.15.47:0/2668916630 >> 172.21.15.47:0/2668916630 conn(0x7fb650103c30 msgr2=0x7fb650106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.251+0000 7fb65810c700 1 -- 172.21.15.47:0/2668916630 shutdown_connections 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.251+0000 7fb65810c700 1 -- 172.21.15.47:0/2668916630 wait complete. 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb65810c700 1 Processor -- start 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb65810c700 1 -- start start 2024-04-25T20:48:52.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb65810c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb65007eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb65810c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb650109d50 con 0x7fb650108670 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb655ea8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb65007eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb655ea8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb65007eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56196/0 (socket says 172.21.15.47:56196) 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb655ea8700 1 -- 172.21.15.47:0/1681398360 learned_addr learned my addr 172.21.15.47:0/1681398360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb655ea8700 1 -- 172.21.15.47:0/1681398360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65007f0b0 con 0x7fb650108670 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.252+0000 7fb655ea8700 1 --2- 172.21.15.47:0/1681398360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb65007eb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb644000c00 tx=0x7fb644004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.253+0000 7fb642ffd700 1 -- 172.21.15.47:0/1681398360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb644003db0 con 0x7fb650108670 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.253+0000 7fb642ffd700 1 -- 172.21.15.47:0/1681398360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb644011630 con 0x7fb650108670 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.253+0000 7fb642ffd700 1 -- 172.21.15.47:0/1681398360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb644011950 con 0x7fb650108670 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.253+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65007d2a0 con 0x7fb650108670 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.253+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65007d7e0 con 0x7fb650108670 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.253+0000 7fb642ffd700 1 -- 172.21.15.47:0/1681398360 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb64401a460 con 0x7fb650108670 2024-04-25T20:48:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.253+0000 7fb642ffd700 1 -- 172.21.15.47:0/1681398360 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb64401b780 con 0x7fb650108670 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.254+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6500640a0 con 0x7fb650108670 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.256+0000 7fb642ffd700 1 -- 172.21.15.47:0/1681398360 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb644031020 con 0x7fb650108670 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.331+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb6501955f0 con 0x7fb650108670 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.331+0000 7fb642ffd700 1 -- 172.21.15.47:0/1681398360 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb64401b9d0 con 0x7fb650108670 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.332+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 msgr2=0x7fb65007eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.332+0000 7fb65810c700 1 --2- 172.21.15.47:0/1681398360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb65007eb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb644000c00 tx=0x7fb644004750 comp rx=0 tx=0).stop 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.332+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 shutdown_connections 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.332+0000 7fb65810c700 1 --2- 172.21.15.47:0/1681398360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb650108670 0x7fb65007eb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.332+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 >> 172.21.15.47:0/1681398360 conn(0x7fb650103c30 msgr2=0x7fb650192930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.332+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 shutdown_connections 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:52.332+0000 7fb65810c700 1 -- 172.21.15.47:0/1681398360 wait complete. 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:mon is available 2024-04-25T20:48:52.950 INFO:teuthology.orchestra.run.smithi047.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T20:48:53.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T20:48:53.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.365+0000 7f7ee07ca700 1 Processor -- start 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.365+0000 7f7ee07ca700 1 -- start start 2024-04-25T20:48:53.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.365+0000 7f7ee07ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed8108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.365+0000 7f7ee07ca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8109060 con 0x7f7ed8108670 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.365+0000 7f7ede566700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed8108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.365+0000 7f7ede566700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed8108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56206/0 (socket says 172.21.15.47:56206) 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.365+0000 7f7ede566700 1 -- 172.21.15.47:0/2880085701 learned_addr learned my addr 172.21.15.47:0/2880085701 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.365+0000 7f7ede566700 1 -- 172.21.15.47:0/2880085701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed8109820 con 0x7f7ed8108670 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7ede566700 1 --2- 172.21.15.47:0/2880085701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed8108a90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40098a0 tx=0x7f7ed4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81aa12c782fba1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7edd564700 1 -- 172.21.15.47:0/2880085701 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ed4004160 con 0x7f7ed8108670 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7edd564700 1 -- 172.21.15.47:0/2880085701 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed4004300 con 0x7f7ed8108670 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7edd564700 1 -- 172.21.15.47:0/2880085701 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ed40036a0 con 0x7f7ed8108670 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7ee07ca700 1 -- 172.21.15.47:0/2880085701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 msgr2=0x7f7ed8108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7ee07ca700 1 --2- 172.21.15.47:0/2880085701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed8108a90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40098a0 tx=0x7f7ed4009bb0 comp rx=0 tx=0).stop 2024-04-25T20:48:53.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7ee07ca700 1 -- 172.21.15.47:0/2880085701 shutdown_connections 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7ee07ca700 1 --2- 172.21.15.47:0/2880085701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed8108a90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7ee07ca700 1 -- 172.21.15.47:0/2880085701 >> 172.21.15.47:0/2880085701 conn(0x7f7ed8103c30 msgr2=0x7f7ed8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7ee07ca700 1 -- 172.21.15.47:0/2880085701 shutdown_connections 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.366+0000 7f7ee07ca700 1 -- 172.21.15.47:0/2880085701 wait complete. 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.367+0000 7f7ee07ca700 1 Processor -- start 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.367+0000 7f7ee07ca700 1 -- start start 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.367+0000 7f7ee07ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed807eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.367+0000 7f7ee07ca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8109d50 con 0x7f7ed8108670 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.367+0000 7f7ede566700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed807eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.367+0000 7f7ede566700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed807eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56218/0 (socket says 172.21.15.47:56218) 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.367+0000 7f7ede566700 1 -- 172.21.15.47:0/1592905859 learned_addr learned my addr 172.21.15.47:0/1592905859 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ede566700 1 -- 172.21.15.47:0/1592905859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed807f0b0 con 0x7f7ed8108670 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ede566700 1 --2- 172.21.15.47:0/1592905859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed807eb70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4000c00 tx=0x7f7ed4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ecf7fe700 1 -- 172.21.15.47:0/1592905859 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ed4003db0 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ecf7fe700 1 -- 172.21.15.47:0/1592905859 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed4011630 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ecf7fe700 1 -- 172.21.15.47:0/1592905859 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ed4011950 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed807d2a0 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed807d7e0 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ecf7fe700 1 -- 172.21.15.47:0/1592905859 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ed401a460 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.368+0000 7f7ecf7fe700 1 -- 172.21.15.47:0/1592905859 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ed401b780 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.369+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed80640a0 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.371+0000 7f7ecf7fe700 1 -- 172.21.15.47:0/1592905859 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7ed4031020 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.442+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7ed8195360 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.445+0000 7f7ecf7fe700 1 -- 172.21.15.47:0/1592905859 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f7ed401b9d0 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.446+0000 7f7ecf7fe700 1 -- 172.21.15.47:0/1592905859 <== mon.0 v2:172.21.15.47:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ed401ad70 con 0x7f7ed8108670 2024-04-25T20:48:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.446+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 msgr2=0x7f7ed807eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.446+0000 7f7ee07ca700 1 --2- 172.21.15.47:0/1592905859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed807eb70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4000c00 tx=0x7f7ed4004750 comp rx=0 tx=0).stop 2024-04-25T20:48:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.446+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 shutdown_connections 2024-04-25T20:48:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.446+0000 7f7ee07ca700 1 --2- 172.21.15.47:0/1592905859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed8108670 0x7f7ed807eb70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.446+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 >> 172.21.15.47:0/1592905859 conn(0x7f7ed8103c30 msgr2=0x7f7ed8192930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.447+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 shutdown_connections 2024-04-25T20:48:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:53.447+0000 7f7ee07ca700 1 -- 172.21.15.47:0/1592905859 wait complete. 2024-04-25T20:48:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:Generating new minimal ceph.conf... 2024-04-25T20:48:54.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:53 smithi047 ceph-mon[28821]: from='client.? 172.21.15.47:0/1592905859' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T20:48:54.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:53 smithi047 ceph-mon[28821]: from='client.? 172.21.15.47:0/1592905859' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T20:48:54.975 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:54 smithi047 ceph-mon[28821]: from='client.? 172.21.15.47:0/2800463154' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:54.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.334+0000 7ff82bcba700 1 Processor -- start 2024-04-25T20:48:54.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.334+0000 7ff82bcba700 1 -- start start 2024-04-25T20:48:54.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.334+0000 7ff82bcba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82410b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:54.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.334+0000 7ff82bcba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff824063d20 con 0x7ff8241090f0 2024-04-25T20:48:54.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.335+0000 7ff829a56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82410b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.335+0000 7ff829a56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82410b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49808/0 (socket says 172.21.15.47:49808) 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.335+0000 7ff829a56700 1 -- 172.21.15.47:0/706679421 learned_addr learned my addr 172.21.15.47:0/706679421 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.335+0000 7ff829a56700 1 -- 172.21.15.47:0/706679421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82410ba20 con 0x7ff8241090f0 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.335+0000 7ff829a56700 1 --2- 172.21.15.47:0/706679421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82410b4e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff8200098a0 tx=0x7ff820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1e1204c1ebe125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.335+0000 7ff828a54700 1 -- 172.21.15.47:0/706679421 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820004160 con 0x7ff8241090f0 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.335+0000 7ff828a54700 1 -- 172.21.15.47:0/706679421 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff820004300 con 0x7ff8241090f0 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.335+0000 7ff828a54700 1 -- 172.21.15.47:0/706679421 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820011af0 con 0x7ff8241090f0 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.336+0000 7ff82bcba700 1 -- 172.21.15.47:0/706679421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 msgr2=0x7ff82410b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.336+0000 7ff82bcba700 1 --2- 172.21.15.47:0/706679421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82410b4e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff8200098a0 tx=0x7ff820009bb0 comp rx=0 tx=0).stop 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.336+0000 7ff82bcba700 1 -- 172.21.15.47:0/706679421 shutdown_connections 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.336+0000 7ff82bcba700 1 --2- 172.21.15.47:0/706679421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82410b4e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.336+0000 7ff82bcba700 1 -- 172.21.15.47:0/706679421 >> 172.21.15.47:0/706679421 conn(0x7ff824102b80 msgr2=0x7ff824104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.336+0000 7ff82bcba700 1 -- 172.21.15.47:0/706679421 shutdown_connections 2024-04-25T20:48:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.336+0000 7ff82bcba700 1 -- 172.21.15.47:0/706679421 wait complete. 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff82bcba700 1 Processor -- start 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff82bcba700 1 -- start start 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff82bcba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff82bcba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff82410c970 con 0x7ff8241090f0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff829a56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff829a56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49818/0 (socket says 172.21.15.47:49818) 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff829a56700 1 -- 172.21.15.47:0/2800463154 learned_addr learned my addr 172.21.15.47:0/2800463154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff829a56700 1 -- 172.21.15.47:0/2800463154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82419ca40 con 0x7ff8241090f0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.337+0000 7ff829a56700 1 --2- 172.21.15.47:0/2800463154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82419c500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff820009fb0 tx=0x7ff820011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.338+0000 7ff81affd700 1 -- 172.21.15.47:0/2800463154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8200036a0 con 0x7ff8241090f0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.338+0000 7ff81affd700 1 -- 172.21.15.47:0/2800463154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff82001a430 con 0x7ff8241090f0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.338+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82419ccd0 con 0x7ff8241090f0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.338+0000 7ff81affd700 1 -- 172.21.15.47:0/2800463154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820019550 con 0x7ff8241090f0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.338+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82419f9c0 con 0x7ff8241090f0 2024-04-25T20:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.338+0000 7ff81affd700 1 -- 172.21.15.47:0/2800463154 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff820019b90 con 0x7ff8241090f0 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.338+0000 7ff81affd700 1 -- 172.21.15.47:0/2800463154 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff820024730 con 0x7ff8241090f0 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.339+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8240640a0 con 0x7ff8241090f0 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.341+0000 7ff81affd700 1 -- 172.21.15.47:0/2800463154 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff820043e50 con 0x7ff8241090f0 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.412+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff82419ff10 con 0x7ff8241090f0 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.412+0000 7ff81affd700 1 -- 172.21.15.47:0/2800463154 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff820032030 con 0x7ff8241090f0 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.413+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 msgr2=0x7ff82419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.413+0000 7ff82bcba700 1 --2- 172.21.15.47:0/2800463154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82419c500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff820009fb0 tx=0x7ff820011940 comp rx=0 tx=0).stop 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.413+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 shutdown_connections 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.413+0000 7ff82bcba700 1 --2- 172.21.15.47:0/2800463154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8241090f0 0x7ff82419c500 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.413+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 >> 172.21.15.47:0/2800463154 conn(0x7ff824102b80 msgr2=0x7ff824103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.413+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 shutdown_connections 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:54.413+0000 7ff82bcba700 1 -- 172.21.15.47:0/2800463154 wait complete. 2024-04-25T20:48:54.985 INFO:teuthology.orchestra.run.smithi047.stdout:Restarting the monitor... 2024-04-25T20:48:55.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:55 smithi047 systemd[1]: Stopping Ceph mon.a for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:48:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a[28817]: 2024-04-25T20:48:55.260+0000 7fea8791e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T20:48:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a[28817]: 2024-04-25T20:48:55.260+0000 7fea8791e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T20:48:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:55 smithi047 podman[29066]: 2024-04-25 20:48:55.456003546 +0000 UTC m=+0.297402924 container died 511e5fe86af03c067d78cefdc7c5d739fda58bbeaa0096e0937bf30fa1a29e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T20:48:55.971 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:55 smithi047 podman[29066]: 2024-04-25 20:48:55.752731372 +0000 UTC m=+0.594130750 container cleanup 511e5fe86af03c067d78cefdc7c5d739fda58bbeaa0096e0937bf30fa1a29e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-25T20:48:55.971 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:55 smithi047 bash[29066]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a 2024-04-25T20:48:56.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:55 smithi047 podman[29080]: 2024-04-25 20:48:55.970343501 +0000 UTC m=+0.510694269 container remove 511e5fe86af03c067d78cefdc7c5d739fda58bbeaa0096e0937bf30fa1a29e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:48:56.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 systemd[1]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T20:48:56.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 systemd[1]: Stopped Ceph mon.a for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:48:56.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 systemd[1]: Starting Ceph mon.a for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:48:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 podman[29176]: 2024-04-25 20:48:56.329539451 +0000 UTC m=+0.105894979 container create 0248222cfd5e30e77fcbe0fa8e4c2e5fd27af1236219a26e29688b0346c64735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T20:48:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 podman[29176]: 2024-04-25 20:48:56.242790046 +0000 UTC m=+0.019145547 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:48:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 podman[29176]: 2024-04-25 20:48:56.534300305 +0000 UTC m=+0.310655804 container init 0248222cfd5e30e77fcbe0fa8e4c2e5fd27af1236219a26e29688b0346c64735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T20:48:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 podman[29176]: 2024-04-25 20:48:56.53734496 +0000 UTC m=+0.313700447 container start 0248222cfd5e30e77fcbe0fa8e4c2e5fd27af1236219a26e29688b0346c64735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T20:48:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T20:48:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T20:48:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: pidfile_write: ignore empty --pid-file 2024-04-25T20:48:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: load: jerasure load: lrc 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: RocksDB version: 7.9.2 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Git sha 0 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: DB SUMMARY 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: DB Session ID: GJF8UDRI2LOKU6AOALUK 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: CURRENT file: CURRENT 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106032 ; 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.error_if_exists: 0 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.create_if_missing: 0 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.paranoid_checks: 1 2024-04-25T20:48:56.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.env: 0x562b2729acc0 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.info_log: 0x562b299a3be0 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.statistics: (nil) 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.use_fsync: 0 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_log_file_size: 0 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T20:48:56.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.allow_fallocate: 1 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.use_direct_reads: 0 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.db_log_dir: 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.wal_dir: 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T20:48:56.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T20:48:56.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T20:48:56.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.write_buffer_manager: 0x562b299370e0 2024-04-25T20:48:56.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T20:48:56.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T20:48:56.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T20:48:56.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T20:48:56.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T20:48:56.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T20:48:56.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T20:48:56.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T20:48:56.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.unordered_write: 0 2024-04-25T20:48:56.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.row_cache: None 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.wal_filter: None 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.two_write_queues: 0 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.wal_compression: 0 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.atomic_flush: 0 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T20:48:56.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.log_readahead_size: 0 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_background_jobs: 2 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_background_compactions: -1 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_subcompactions: 1 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T20:48:56.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_open_files: -1 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_background_flushes: -1 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Compression algorithms supported: 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T20:48:56.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: kZSTD supported: 0 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: kXpressCompression supported: 0 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: kZlibCompression supported: 1 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: kSnappyCompression supported: 1 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: kLZ4Compression supported: 1 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: kBZip2Compression supported: 0 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.merge_operator: 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_filter: None 2024-04-25T20:48:56.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562b299a3dc0) 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks: 1 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_top_level_index_and_filter: 1 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_type: 0 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_index_type: 0 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_shortening: 1 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T20:48:56.620 INFO:journalctl@ceph.mon.a.smithi047.stdout: checksum: 4 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: no_block_cache: 0 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache: 0x562b299691f0 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_name: BinnedLRUCache 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_options: 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: capacity : 536870912 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_shard_bits : 4 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: strict_capacity_limit : 0 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: high_pri_pool_ratio: 0.000 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_compressed: (nil) 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: persistent_cache: (nil) 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size: 4096 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size_deviation: 10 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_restart_interval: 16 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_block_restart_interval: 1 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: metadata_block_size: 4096 2024-04-25T20:48:56.621 INFO:journalctl@ceph.mon.a.smithi047.stdout: partition_filters: 0 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: use_delta_encoding: 1 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: filter_policy: bloomfilter 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: whole_key_filtering: 1 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: verify_compression: 0 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: read_amp_bytes_per_bit: 0 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: format_version: 5 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_index_compression: 1 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_align: 0 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: max_auto_readahead_size: 262144 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: prepopulate_block_cache: 0 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: initial_auto_readahead_size: 8192 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression: NoCompression 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.num_levels: 7 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:48:56.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T20:48:56.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T20:48:56.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T20:48:56.626 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.inplace_update_support: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.bloom_locality: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.max_successive_merges: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.ttl: 2592000 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T20:48:56.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.enable_blob_files: false 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.min_blob_size: 0 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T20:48:56.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8e33405d-3e2f-45c7-a696-0c2d2f939ed5 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078136565985, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078136567071, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98973, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91152, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714078136, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8e33405d-3e2f-45c7-a696-0c2d2f939ed5", "db_session_id": "GJF8UDRI2LOKU6AOALUK", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078136567155, "job": 1, "event": "recovery_finished"} 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562b29a7e000 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: DB pointer 0x562b29a68000 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** DB Stats ** 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:48:56.629 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: L0 2/0 100.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: Sum 2/0 100.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T20:48:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative compaction: 0.00 GB write, 30.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval compaction: 0.00 GB write, 30.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache BinnedLRUCache@0x562b299691f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T20:48:56.631 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: starting mon.a rank 0 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a@-1(???) e1 preinit fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a@-1(???).mds e1 new map 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a@-1(???).mds e1 print_map 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout: e1 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout: legacy client fscid: -1 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout: No filesystems configured 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:48:56.632 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T20:48:56.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 bash[29176]: 0248222cfd5e30e77fcbe0fa8e4c2e5fd27af1236219a26e29688b0346c64735 2024-04-25T20:48:56.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 systemd[1]: Started Ceph mon.a for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:48:56.633 INFO:teuthology.orchestra.run.smithi047.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T20:48:56.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T20:48:56.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: monmap epoch 1 2024-04-25T20:48:56.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:48:56.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: last_changed 2024-04-25T20:48:48.454006+0000 2024-04-25T20:48:56.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:48:56.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T20:48:56.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: election_strategy: 1 2024-04-25T20:48:56.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:48:56.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: fsmap 2024-04-25T20:48:56.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T20:48:56.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:56 smithi047 ceph-mon[29195]: mgrmap e1: no daemons active 2024-04-25T20:48:57.706 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.049+0000 7f8615915700 1 Processor -- start 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.049+0000 7f8615915700 1 -- start start 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.050+0000 7f8615915700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f8610108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.050+0000 7f8615915700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8610109060 con 0x7f8610108670 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.050+0000 7f860effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f8610108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.050+0000 7f860effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f8610108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49834/0 (socket says 172.21.15.47:49834) 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.050+0000 7f860effd700 1 -- 172.21.15.47:0/2656065789 learned_addr learned my addr 172.21.15.47:0/2656065789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.050+0000 7f860effd700 1 -- 172.21.15.47:0/2656065789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8610109820 con 0x7f8610108670 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.050+0000 7f860effd700 1 --2- 172.21.15.47:0/2656065789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f8610108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86000098a0 tx=0x7f8600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6984518e35fcf186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f860dffb700 1 -- 172.21.15.47:0/2656065789 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8600004160 con 0x7f8610108670 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f860dffb700 1 -- 172.21.15.47:0/2656065789 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8600004300 con 0x7f8610108670 2024-04-25T20:48:57.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f860dffb700 1 -- 172.21.15.47:0/2656065789 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8600011af0 con 0x7f8610108670 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f8615915700 1 -- 172.21.15.47:0/2656065789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 msgr2=0x7f8610108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f8615915700 1 --2- 172.21.15.47:0/2656065789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f8610108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86000098a0 tx=0x7f8600009bb0 comp rx=0 tx=0).stop 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f8615915700 1 -- 172.21.15.47:0/2656065789 shutdown_connections 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f8615915700 1 --2- 172.21.15.47:0/2656065789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f8610108a90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f8615915700 1 -- 172.21.15.47:0/2656065789 >> 172.21.15.47:0/2656065789 conn(0x7f8610103c30 msgr2=0x7f8610106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f8615915700 1 -- 172.21.15.47:0/2656065789 shutdown_connections 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.051+0000 7f8615915700 1 -- 172.21.15.47:0/2656065789 wait complete. 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.052+0000 7f8615915700 1 Processor -- start 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.052+0000 7f8615915700 1 -- start start 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.052+0000 7f8615915700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f861007eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.052+0000 7f8615915700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f861010ca50 con 0x7f8610108670 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f860effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f861007eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f860effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f861007eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49838/0 (socket says 172.21.15.47:49838) 2024-04-25T20:48:57.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f860effd700 1 -- 172.21.15.47:0/712453981 learned_addr learned my addr 172.21.15.47:0/712453981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f860effd700 1 -- 172.21.15.47:0/712453981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861007f0b0 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f860effd700 1 --2- 172.21.15.47:0/712453981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f861007eb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f86000117b0 tx=0x7f8600011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f8614913700 1 -- 172.21.15.47:0/712453981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8600003af0 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f8614913700 1 -- 172.21.15.47:0/712453981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8600020070 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f8614913700 1 -- 172.21.15.47:0/712453981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8600019630 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861007d2a0 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861007d7e0 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.053+0000 7f8614913700 1 -- 172.21.15.47:0/712453981 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8600019c80 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.054+0000 7f8614913700 1 -- 172.21.15.47:0/712453981 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f860000b720 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.054+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861010cbd0 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.057+0000 7f8614913700 1 -- 172.21.15.47:0/712453981 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f86000042a0 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.128+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f86100640a0 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.130+0000 7f8614913700 1 -- 172.21.15.47:0/712453981 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f860001a9c0 con 0x7f8610108670 2024-04-25T20:48:57.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.131+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 msgr2=0x7f861007eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:57.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.131+0000 7f8615915700 1 --2- 172.21.15.47:0/712453981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f861007eb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f86000117b0 tx=0x7f8600011b70 comp rx=0 tx=0).stop 2024-04-25T20:48:57.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.131+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 shutdown_connections 2024-04-25T20:48:57.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.131+0000 7f8615915700 1 --2- 172.21.15.47:0/712453981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8610108670 0x7f861007eb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:57.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.131+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 >> 172.21.15.47:0/712453981 conn(0x7f8610103c30 msgr2=0x7f8610104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:57.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.131+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 shutdown_connections 2024-04-25T20:48:57.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:57.131+0000 7f8615915700 1 -- 172.21.15.47:0/712453981 wait complete. 2024-04-25T20:48:57.710 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T20:48:57.741 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:48:57.741 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mgr... 2024-04-25T20:48:57.741 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T20:48:57.741 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T20:48:58.007 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.x 2024-04-25T20:48:58.007 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T20:48:58.162 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ca8de06-0345-11ef-bc93-c7b262605968.target.wants/ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0ca8de06-0345-11ef-bc93-c7b262605968@.service. 2024-04-25T20:48:58.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:58 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/712453981' entity='client.admin' 2024-04-25T20:48:58.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 systemd[1]: Starting Ceph mgr.x for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:48:58.670 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 podman[29407]: 2024-04-25 20:48:58.401526982 +0000 UTC m=+0.110664610 container create 9dba1d645dcfa0045ce2a63aa2366b63b8e210899e902c97756cecb337a47f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T20:48:58.670 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 podman[29407]: 2024-04-25 20:48:58.309351897 +0000 UTC m=+0.018489512 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:48:58.671 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 podman[29407]: 2024-04-25 20:48:58.595240173 +0000 UTC m=+0.304377759 container init 9dba1d645dcfa0045ce2a63aa2366b63b8e210899e902c97756cecb337a47f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T20:48:58.671 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 podman[29407]: 2024-04-25 20:48:58.597762233 +0000 UTC m=+0.306899828 container start 9dba1d645dcfa0045ce2a63aa2366b63b8e210899e902c97756cecb337a47f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:48:58.671 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 bash[29407]: 9dba1d645dcfa0045ce2a63aa2366b63b8e210899e902c97756cecb337a47f5f 2024-04-25T20:48:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:48:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-04-25T20:48:58.696 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:48:58.696 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-25T20:48:58.696 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-04-25T20:48:58.696 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T20:48:58.706 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:48:58.707 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-04-25T20:48:58.715 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:48:58.716 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-25T20:48:58.716 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-04-25T20:48:58.716 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T20:48:58.716 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr to start... 2024-04-25T20:48:58.716 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr... 2024-04-25T20:48:58.942 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 systemd[1]: Started Ceph mgr.x for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:48:58.942 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:48:58.766+0000 7f761da21200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:48:59.265 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:48:58.965+0000 7f761da21200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:48:59.265 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:48:59.071+0000 7f761da21200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:48:59.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:48:59 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2869462121' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:48:59.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:48:59.405+0000 7f761da21200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:48:59.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:48:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:48:59.492+0000 7f761da21200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:48:59.843 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:48:59.843 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:48:59.843 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:48:59.843 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:48:59.843 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:48:59.843 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:48:59.843 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:48:59.843 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:48:59.844 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:48:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:48:59.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:48:59.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:48:51.767189+0000", 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.183+0000 7efc1b1be700 1 Processor -- start 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.183+0000 7efc1b1be700 1 -- start start 2024-04-25T20:48:59.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.183+0000 7efc1b1be700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14106d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.183+0000 7efc1b1be700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efc14107300 con 0x7efc14106910 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.183+0000 7efc18f5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14106d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.183+0000 7efc18f5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14106d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49864/0 (socket says 172.21.15.47:49864) 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.183+0000 7efc18f5a700 1 -- 172.21.15.47:0/4276148866 learned_addr learned my addr 172.21.15.47:0/4276148866 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.184+0000 7efc18f5a700 1 -- 172.21.15.47:0/4276148866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc141098f0 con 0x7efc14106910 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.184+0000 7efc18f5a700 1 --2- 172.21.15.47:0/4276148866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14106d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efc080098a0 tx=0x7efc08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c0665613fd4673 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.184+0000 7efc137fe700 1 -- 172.21.15.47:0/4276148866 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc08004160 con 0x7efc14106910 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.184+0000 7efc137fe700 1 -- 172.21.15.47:0/4276148866 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efc08004300 con 0x7efc14106910 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.184+0000 7efc137fe700 1 -- 172.21.15.47:0/4276148866 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc08011af0 con 0x7efc14106910 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.185+0000 7efc1b1be700 1 -- 172.21.15.47:0/4276148866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 msgr2=0x7efc14106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:59.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.185+0000 7efc1b1be700 1 --2- 172.21.15.47:0/4276148866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14106d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efc080098a0 tx=0x7efc08009bb0 comp rx=0 tx=0).stop 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.185+0000 7efc1b1be700 1 -- 172.21.15.47:0/4276148866 shutdown_connections 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.185+0000 7efc1b1be700 1 --2- 172.21.15.47:0/4276148866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14106d30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.185+0000 7efc1b1be700 1 -- 172.21.15.47:0/4276148866 >> 172.21.15.47:0/4276148866 conn(0x7efc141022b0 msgr2=0x7efc141046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.185+0000 7efc1b1be700 1 -- 172.21.15.47:0/4276148866 shutdown_connections 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.185+0000 7efc1b1be700 1 -- 172.21.15.47:0/4276148866 wait complete. 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc1b1be700 1 Processor -- start 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc1b1be700 1 -- start start 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc1b1be700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc1b1be700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efc1410cb20 con 0x7efc14106910 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc18f5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:59.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc18f5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49878/0 (socket says 172.21.15.47:49878) 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc18f5a700 1 -- 172.21.15.47:0/2869462121 learned_addr learned my addr 172.21.15.47:0/2869462121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc18f5a700 1 -- 172.21.15.47:0/2869462121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc141096b0 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.186+0000 7efc18f5a700 1 --2- 172.21.15.47:0/2869462121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14109170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc080117b0 tx=0x7efc08011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.187+0000 7efc11ffb700 1 -- 172.21.15.47:0/2869462121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc08003af0 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.187+0000 7efc11ffb700 1 -- 172.21.15.47:0/2869462121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efc08020070 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.187+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc141078a0 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.187+0000 7efc11ffb700 1 -- 172.21.15.47:0/2869462121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc08019630 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.187+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc14107de0 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.187+0000 7efc11ffb700 1 -- 172.21.15.47:0/2869462121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efc08019c30 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.187+0000 7efc11ffb700 1 -- 172.21.15.47:0/2869462121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efc0800b720 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.188+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc140640a0 con 0x7efc14106910 2024-04-25T20:48:59.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.190+0000 7efc11ffb700 1 -- 172.21.15.47:0/2869462121 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efc080042a0 con 0x7efc14106910 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.264+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efc14108090 con 0x7efc14106910 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.264+0000 7efc11ffb700 1 -- 172.21.15.47:0/2869462121 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7efc0801a9c0 con 0x7efc14106910 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.266+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 msgr2=0x7efc14109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.266+0000 7efc1b1be700 1 --2- 172.21.15.47:0/2869462121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14109170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc080117b0 tx=0x7efc08011b70 comp rx=0 tx=0).stop 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.266+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 shutdown_connections 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.266+0000 7efc1b1be700 1 --2- 172.21.15.47:0/2869462121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc14106910 0x7efc14109170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.266+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 >> 172.21.15.47:0/2869462121 conn(0x7efc141022b0 msgr2=0x7efc14102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.266+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 shutdown_connections 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:48:59.266+0000 7efc1b1be700 1 -- 172.21.15.47:0/2869462121 wait complete. 2024-04-25T20:48:59.852 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (1/15)... 2024-04-25T20:49:01.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:00.763+0000 7f761da21200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:49:01.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:00.938+0000 7f761da21200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:49:01.417 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:01.113+0000 7f761da21200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:49:01.417 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:01.416+0000 7f761da21200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:49:01.730 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:01.513+0000 7f761da21200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:49:01.731 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:01.729+0000 7f761da21200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:49:01.981 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:01.932+0000 7f761da21200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:49:02.592 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:02 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/156356548' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:49:02.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:02.591+0000 7f761da21200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:49:02.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:02.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:49:02.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:49:02.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:49:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:48:51.767189+0000", 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.298+0000 7f88fa9ae700 1 Processor -- start 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.298+0000 7f88fa9ae700 1 -- start start 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.298+0000 7f88fa9ae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f410b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:02.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.298+0000 7f88fa9ae700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f88f407ce30 con 0x7f88f41090e0 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.298+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f410b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.298+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f410b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49892/0 (socket says 172.21.15.47:49892) 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.298+0000 7f88f3fff700 1 -- 172.21.15.47:0/1918082901 learned_addr learned my addr 172.21.15.47:0/1918082901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.299+0000 7f88f3fff700 1 -- 172.21.15.47:0/1918082901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f410ba10 con 0x7f88f41090e0 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.299+0000 7f88f3fff700 1 --2- 172.21.15.47:0/1918082901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f410b4d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88e80098a0 tx=0x7f88e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a814c7f93905ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.299+0000 7f88f2ffd700 1 -- 172.21.15.47:0/1918082901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e8004160 con 0x7f88f41090e0 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.299+0000 7f88f2ffd700 1 -- 172.21.15.47:0/1918082901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f88e8004300 con 0x7f88f41090e0 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.299+0000 7f88f2ffd700 1 -- 172.21.15.47:0/1918082901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e8011af0 con 0x7f88f41090e0 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 -- 172.21.15.47:0/1918082901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 msgr2=0x7f88f410b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 --2- 172.21.15.47:0/1918082901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f410b4d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88e80098a0 tx=0x7f88e8009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 -- 172.21.15.47:0/1918082901 shutdown_connections 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 --2- 172.21.15.47:0/1918082901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f410b4d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:02.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 -- 172.21.15.47:0/1918082901 >> 172.21.15.47:0/1918082901 conn(0x7f88f4102af0 msgr2=0x7f88f4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 -- 172.21.15.47:0/1918082901 shutdown_connections 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 -- 172.21.15.47:0/1918082901 wait complete. 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 Processor -- start 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.300+0000 7f88fa9ae700 1 -- start start 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.301+0000 7f88fa9ae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.301+0000 7f88fa9ae700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f88f410c900 con 0x7f88f41090e0 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.301+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.301+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49908/0 (socket says 172.21.15.47:49908) 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.301+0000 7f88f3fff700 1 -- 172.21.15.47:0/156356548 learned_addr learned my addr 172.21.15.47:0/156356548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.301+0000 7f88f3fff700 1 -- 172.21.15.47:0/156356548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f419eca0 con 0x7f88f41090e0 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.301+0000 7f88f3fff700 1 --2- 172.21.15.47:0/156356548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f419e760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009fb0 tx=0x7f88e8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.302+0000 7f88f17fa700 1 -- 172.21.15.47:0/156356548 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e80036a0 con 0x7f88f41090e0 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.302+0000 7f88f17fa700 1 -- 172.21.15.47:0/156356548 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f88e801a430 con 0x7f88f41090e0 2024-04-25T20:49:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.302+0000 7f88f17fa700 1 -- 172.21.15.47:0/156356548 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e8019550 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.302+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f419ef30 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.302+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f41a1c20 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.302+0000 7f88f17fa700 1 -- 172.21.15.47:0/156356548 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f88e8019b90 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.302+0000 7f88f17fa700 1 -- 172.21.15.47:0/156356548 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f88e8024730 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.303+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f40640a0 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.305+0000 7f88f17fa700 1 -- 172.21.15.47:0/156356548 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f88e8043e50 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.397+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f88f41a1df0 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.398+0000 7f88f17fa700 1 -- 172.21.15.47:0/156356548 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f88e8032030 con 0x7f88f41090e0 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.399+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 msgr2=0x7f88f419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.399+0000 7f88fa9ae700 1 --2- 172.21.15.47:0/156356548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f419e760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009fb0 tx=0x7f88e8011940 comp rx=0 tx=0).stop 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.399+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 shutdown_connections 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.399+0000 7f88fa9ae700 1 --2- 172.21.15.47:0/156356548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88f41090e0 0x7f88f419e760 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.399+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 >> 172.21.15.47:0/156356548 conn(0x7f88f4102af0 msgr2=0x7f88f4103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:02.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.399+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 shutdown_connections 2024-04-25T20:49:02.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:02.399+0000 7f88fa9ae700 1 -- 172.21.15.47:0/156356548 wait complete. 2024-04-25T20:49:02.988 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (2/15)... 2024-04-25T20:49:03.344 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:03.035+0000 7f761da21200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:49:03.344 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:03.134+0000 7f761da21200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:49:03.344 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:03.343+0000 7f761da21200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:49:04.236 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:03.888+0000 7f761da21200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:49:04.236 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:03.983+0000 7f761da21200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:49:04.592 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:04.235+0000 7f761da21200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:49:04.592 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:04.373+0000 7f761da21200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:49:04.593 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:04.471+0000 7f761da21200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:49:04.593 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:04.591+0000 7f761da21200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:49:05.174 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:04.903+0000 7f761da21200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:49:05.779 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:05 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2018305158' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:49:05.779 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:05 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:05.552+0000 7f761da21200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:49:05.779 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:05 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:05.778+0000 7f761da21200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:49:06.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:49:06.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:49:06.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:06.053 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:49:06.054 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:06.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:06.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:48:51.767189+0000", 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.369+0000 7f780ea86700 1 Processor -- start 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.369+0000 7f780ea86700 1 -- start start 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.369+0000 7f780ea86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f7808108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.369+0000 7f780ea86700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7808109050 con 0x7f7808108660 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.369+0000 7f780c822700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f7808108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.369+0000 7f780c822700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f7808108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38358/0 (socket says 172.21.15.47:38358) 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.369+0000 7f780c822700 1 -- 172.21.15.47:0/1723425053 learned_addr learned my addr 172.21.15.47:0/1723425053 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.369+0000 7f780c822700 1 -- 172.21.15.47:0/1723425053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7808109810 con 0x7f7808108660 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.370+0000 7f780c822700 1 --2- 172.21.15.47:0/1723425053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f7808108a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0098a0 tx=0x7f77fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d5d6b3f2af89a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.370+0000 7f78077fe700 1 -- 172.21.15.47:0/1723425053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77fc004160 con 0x7f7808108660 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.370+0000 7f78077fe700 1 -- 172.21.15.47:0/1723425053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f77fc004300 con 0x7f7808108660 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.370+0000 7f78077fe700 1 -- 172.21.15.47:0/1723425053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77fc011af0 con 0x7f7808108660 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.371+0000 7f780ea86700 1 -- 172.21.15.47:0/1723425053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 msgr2=0x7f7808108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.371+0000 7f780ea86700 1 --2- 172.21.15.47:0/1723425053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f7808108a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0098a0 tx=0x7f77fc009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.371+0000 7f780ea86700 1 -- 172.21.15.47:0/1723425053 shutdown_connections 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.371+0000 7f780ea86700 1 --2- 172.21.15.47:0/1723425053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f7808108a80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.371+0000 7f780ea86700 1 -- 172.21.15.47:0/1723425053 >> 172.21.15.47:0/1723425053 conn(0x7f7808103c40 msgr2=0x7f7808106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:06.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.371+0000 7f780ea86700 1 -- 172.21.15.47:0/1723425053 shutdown_connections 2024-04-25T20:49:06.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.371+0000 7f780ea86700 1 -- 172.21.15.47:0/1723425053 wait complete. 2024-04-25T20:49:06.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780ea86700 1 Processor -- start 2024-04-25T20:49:06.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780ea86700 1 -- start start 2024-04-25T20:49:06.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780ea86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f780819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:06.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780ea86700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f780810ca40 con 0x7f7808108660 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780c822700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f780819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780c822700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f780819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38362/0 (socket says 172.21.15.47:38362) 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780c822700 1 -- 172.21.15.47:0/2018305158 learned_addr learned my addr 172.21.15.47:0/2018305158 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780c822700 1 -- 172.21.15.47:0/2018305158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780819ca60 con 0x7f7808108660 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.372+0000 7f780c822700 1 --2- 172.21.15.47:0/2018305158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f780819c520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009fb0 tx=0x7f77fc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.373+0000 7f7805ffb700 1 -- 172.21.15.47:0/2018305158 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77fc0036a0 con 0x7f7808108660 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.373+0000 7f7805ffb700 1 -- 172.21.15.47:0/2018305158 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f77fc01a430 con 0x7f7808108660 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.373+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780819ccf0 con 0x7f7808108660 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.373+0000 7f7805ffb700 1 -- 172.21.15.47:0/2018305158 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77fc019550 con 0x7f7808108660 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.373+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780819f9e0 con 0x7f7808108660 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.373+0000 7f7805ffb700 1 -- 172.21.15.47:0/2018305158 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77fc019b90 con 0x7f7808108660 2024-04-25T20:49:06.060 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.373+0000 7f7805ffb700 1 -- 172.21.15.47:0/2018305158 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77fc024730 con 0x7f7808108660 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.374+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780810cbc0 con 0x7f7808108660 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.376+0000 7f7805ffb700 1 -- 172.21.15.47:0/2018305158 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f77fc043e50 con 0x7f7808108660 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.450+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f780819ff10 con 0x7f7808108660 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.451+0000 7f7805ffb700 1 -- 172.21.15.47:0/2018305158 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f77fc032020 con 0x7f7808108660 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.451+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 msgr2=0x7f780819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.451+0000 7f780ea86700 1 --2- 172.21.15.47:0/2018305158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f780819c520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009fb0 tx=0x7f77fc011940 comp rx=0 tx=0).stop 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.452+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 shutdown_connections 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.452+0000 7f780ea86700 1 --2- 172.21.15.47:0/2018305158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7808108660 0x7f780819c520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.452+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 >> 172.21.15.47:0/2018305158 conn(0x7f7808103c40 msgr2=0x7f7808104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.452+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 shutdown_connections 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:05.452+0000 7f780ea86700 1 -- 172.21.15.47:0/2018305158 wait complete. 2024-04-25T20:49:06.061 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (3/15)... 2024-04-25T20:49:06.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: Activating manager daemon x 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: mgrmap e2: x(active, starting, since 0.00305299s) 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' 2024-04-25T20:49:06.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:06 smithi047 ceph-mon[29195]: from='mgr.14100 172.21.15.47:0/833624064' entity='mgr.x' 2024-04-25T20:49:08.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:07 smithi047 ceph-mon[29195]: mgrmap e3: x(active, since 1.00703s) 2024-04-25T20:49:09.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:08 smithi047 ceph-mon[29195]: mgrmap e4: x(active, since 2s) 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:49:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:49:09.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:49:09.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:49:09.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:49:09.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:49:09.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:49:09.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:09.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:49:09.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:49:09.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:48:51.767189+0000", 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.517+0000 7fe57ea91700 1 Processor -- start 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.517+0000 7fe57ea91700 1 -- start start 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.517+0000 7fe57ea91700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe578108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.517+0000 7fe57ea91700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe578109050 con 0x7fe578108660 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.518+0000 7fe57c82d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe578108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.518+0000 7fe57c82d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe578108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38434/0 (socket says 172.21.15.47:38434) 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.518+0000 7fe57c82d700 1 -- 172.21.15.47:0/3978024472 learned_addr learned my addr 172.21.15.47:0/3978024472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.518+0000 7fe57c82d700 1 -- 172.21.15.47:0/3978024472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe578109810 con 0x7fe578108660 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.518+0000 7fe57c82d700 1 --2- 172.21.15.47:0/3978024472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe578108a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0098a0 tx=0x7fe56c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77904bd8e687ba9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:09.422 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.518+0000 7fe5777fe700 1 -- 172.21.15.47:0/3978024472 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe56c004160 con 0x7fe578108660 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.518+0000 7fe5777fe700 1 -- 172.21.15.47:0/3978024472 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe56c004300 con 0x7fe578108660 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.518+0000 7fe5777fe700 1 -- 172.21.15.47:0/3978024472 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe56c011af0 con 0x7fe578108660 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.519+0000 7fe57ea91700 1 -- 172.21.15.47:0/3978024472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 msgr2=0x7fe578108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.519+0000 7fe57ea91700 1 --2- 172.21.15.47:0/3978024472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe578108a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0098a0 tx=0x7fe56c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.519+0000 7fe57ea91700 1 -- 172.21.15.47:0/3978024472 shutdown_connections 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.519+0000 7fe57ea91700 1 --2- 172.21.15.47:0/3978024472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe578108a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.519+0000 7fe57ea91700 1 -- 172.21.15.47:0/3978024472 >> 172.21.15.47:0/3978024472 conn(0x7fe578103c40 msgr2=0x7fe578106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.519+0000 7fe57ea91700 1 -- 172.21.15.47:0/3978024472 shutdown_connections 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.519+0000 7fe57ea91700 1 -- 172.21.15.47:0/3978024472 wait complete. 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.520+0000 7fe57ea91700 1 Processor -- start 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.520+0000 7fe57ea91700 1 -- start start 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.520+0000 7fe57ea91700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe57807eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.520+0000 7fe57ea91700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe57810ca40 con 0x7fe578108660 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.520+0000 7fe57c82d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe57807eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.520+0000 7fe57c82d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe57807eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38442/0 (socket says 172.21.15.47:38442) 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.520+0000 7fe57c82d700 1 -- 172.21.15.47:0/1237057268 learned_addr learned my addr 172.21.15.47:0/1237057268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.520+0000 7fe57c82d700 1 -- 172.21.15.47:0/1237057268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57807f0b0 con 0x7fe578108660 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.521+0000 7fe57c82d700 1 --2- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe57807eb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009fb0 tx=0x7fe56c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.521+0000 7fe575ffb700 1 -- 172.21.15.47:0/1237057268 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe56c0036a0 con 0x7fe578108660 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.521+0000 7fe575ffb700 1 -- 172.21.15.47:0/1237057268 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe56c01a430 con 0x7fe578108660 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.521+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57807d2a0 con 0x7fe578108660 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.521+0000 7fe575ffb700 1 -- 172.21.15.47:0/1237057268 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe56c019550 con 0x7fe578108660 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.521+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57807d7e0 con 0x7fe578108660 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.522+0000 7fe575ffb700 1 -- 172.21.15.47:0/1237057268 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fe56c019800 con 0x7fe578108660 2024-04-25T20:49:09.424 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.522+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5780640a0 con 0x7fe578108660 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.522+0000 7fe575ffb700 1 --2- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7fe568039560 0x7fe56803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.522+0000 7fe575ffb700 1 -- 172.21.15.47:0/1237057268 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe56c04d880 con 0x7fe578108660 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.523+0000 7fe577fff700 1 --2- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7fe568039560 0x7fe56803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.523+0000 7fe577fff700 1 --2- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7fe568039560 0x7fe56803ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe564006fd0 tx=0x7fe564006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.528+0000 7fe575ffb700 1 -- 172.21.15.47:0/1237057268 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe56c04d2a0 con 0x7fe578108660 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.827+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe57807da60 con 0x7fe578108660 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.827+0000 7fe575ffb700 1 -- 172.21.15.47:0/1237057268 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe56c00fda0 con 0x7fe578108660 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.830+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7fe568039560 msgr2=0x7fe56803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.830+0000 7fe57ea91700 1 --2- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7fe568039560 0x7fe56803ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe564006fd0 tx=0x7fe564006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:09.425 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.830+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 msgr2=0x7fe57807eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:09.426 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.830+0000 7fe57ea91700 1 --2- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe57807eb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009fb0 tx=0x7fe56c011940 comp rx=0 tx=0).stop 2024-04-25T20:49:09.426 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.831+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 shutdown_connections 2024-04-25T20:49:09.426 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.831+0000 7fe57ea91700 1 --2- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7fe568039560 0x7fe56803ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:09.426 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.831+0000 7fe57ea91700 1 --2- 172.21.15.47:0/1237057268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe578108660 0x7fe57807eb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:09.426 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.831+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 >> 172.21.15.47:0/1237057268 conn(0x7fe578103c40 msgr2=0x7fe578104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:09.426 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.831+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 shutdown_connections 2024-04-25T20:49:09.426 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:08.831+0000 7fe57ea91700 1 -- 172.21.15.47:0/1237057268 wait complete. 2024-04-25T20:49:09.426 INFO:teuthology.orchestra.run.smithi047.stdout:mgr is available 2024-04-25T20:49:09.841 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:09 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1237057268' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:49:10.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:49:10.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T20:49:10.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:49:10.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T20:49:10.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T20:49:10.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T20:49:10.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.854+0000 7f1128c7c700 1 Processor -- start 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.854+0000 7f1128c7c700 1 -- start start 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.854+0000 7f1128c7c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f1124108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.854+0000 7f1128c7c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1124109060 con 0x7f1124108670 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.854+0000 7f112259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f1124108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.854+0000 7f112259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f1124108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38452/0 (socket says 172.21.15.47:38452) 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.854+0000 7f112259c700 1 -- 172.21.15.47:0/1148023558 learned_addr learned my addr 172.21.15.47:0/1148023558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.854+0000 7f112259c700 1 -- 172.21.15.47:0/1148023558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1124109820 con 0x7f1124108670 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.855+0000 7f112259c700 1 --2- 172.21.15.47:0/1148023558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f1124108a90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11140098a0 tx=0x7f1114009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54014f9cf41fb726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.855+0000 7f112159a700 1 -- 172.21.15.47:0/1148023558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1114004160 con 0x7f1124108670 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.855+0000 7f112159a700 1 -- 172.21.15.47:0/1148023558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1114004300 con 0x7f1124108670 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.855+0000 7f112159a700 1 -- 172.21.15.47:0/1148023558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1114011af0 con 0x7f1124108670 2024-04-25T20:49:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.855+0000 7f1128c7c700 1 -- 172.21.15.47:0/1148023558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 msgr2=0x7f1124108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.855+0000 7f1128c7c700 1 --2- 172.21.15.47:0/1148023558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f1124108a90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11140098a0 tx=0x7f1114009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 -- 172.21.15.47:0/1148023558 shutdown_connections 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 --2- 172.21.15.47:0/1148023558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f1124108a90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 -- 172.21.15.47:0/1148023558 >> 172.21.15.47:0/1148023558 conn(0x7f1124103c30 msgr2=0x7f1124106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 -- 172.21.15.47:0/1148023558 shutdown_connections 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 -- 172.21.15.47:0/1148023558 wait complete. 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 Processor -- start 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 -- start start 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f112419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.856+0000 7f1128c7c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f112410ca50 con 0x7f1124108670 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f112259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f112419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f112259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f112419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38464/0 (socket says 172.21.15.47:38464) 2024-04-25T20:49:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f112259c700 1 -- 172.21.15.47:0/1925104175 learned_addr learned my addr 172.21.15.47:0/1925104175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f112259c700 1 -- 172.21.15.47:0/1925104175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112419ecf0 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f112259c700 1 --2- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f112419e7b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1114009fb0 tx=0x7f1114011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f11137fe700 1 -- 172.21.15.47:0/1925104175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11140036a0 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112419ef80 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f11137fe700 1 -- 172.21.15.47:0/1925104175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f111401a430 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f11137fe700 1 -- 172.21.15.47:0/1925104175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1114019550 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.857+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11241a1c70 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.858+0000 7f11137fe700 1 -- 172.21.15.47:0/1925104175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f1114019850 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.858+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f112410cbd0 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.858+0000 7f11137fe700 1 --2- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f110c039510 0x7f110c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.858+0000 7f11137fe700 1 -- 172.21.15.47:0/1925104175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f111404d8b0 con 0x7f1124108670 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.859+0000 7f1121d9b700 1 --2- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f110c039510 0x7f110c03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.859+0000 7f1121d9b700 1 --2- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f110c039510 0x7f110c03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1118006fd0 tx=0x7f1118006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:10.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:09.864+0000 7f11137fe700 1 -- 172.21.15.47:0/1925104175 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1114020240 con 0x7f1124108670 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.070+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f11241a1e40 con 0x7f1124108670 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.071+0000 7f11137fe700 1 -- 172.21.15.47:0/1925104175 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f111401a690 con 0x7f1124108670 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f110c039510 msgr2=0x7f110c03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 --2- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f110c039510 0x7f110c03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1118006fd0 tx=0x7f1118006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 msgr2=0x7f112419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 --2- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f112419e7b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1114009fb0 tx=0x7f1114011940 comp rx=0 tx=0).stop 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 shutdown_connections 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 --2- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f110c039510 0x7f110c03b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 --2- 172.21.15.47:0/1925104175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1124108670 0x7f112419e7b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 >> 172.21.15.47:0/1925104175 conn(0x7f1124103c30 msgr2=0x7f1124104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 shutdown_connections 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:10.073+0000 7f1128c7c700 1 -- 172.21.15.47:0/1925104175 wait complete. 2024-04-25T20:49:10.642 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling cephadm module... 2024-04-25T20:49:11.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:10 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1925104175' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T20:49:12.100 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:11 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2347584698' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T20:49:12.100 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:11 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: ignoring --setuser ceph since I am not root 2024-04-25T20:49:12.100 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:11 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: ignoring --setgroup ceph since I am not root 2024-04-25T20:49:12.100 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:11 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:11.971+0000 7fcb8f181200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:49:12.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:12.165+0000 7fcb8f181200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:49:12.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:12.271+0000 7fcb8f181200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:49:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.072+0000 7f1f96047700 1 Processor -- start 2024-04-25T20:49:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.072+0000 7f1f96047700 1 -- start start 2024-04-25T20:49:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.072+0000 7f1f96047700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f90108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.072+0000 7f1f96047700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90109060 con 0x7f1f90108670 2024-04-25T20:49:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.072+0000 7f1f8f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f90108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.072+0000 7f1f8f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f90108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38466/0 (socket says 172.21.15.47:38466) 2024-04-25T20:49:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.072+0000 7f1f8f7fe700 1 -- 172.21.15.47:0/2396844451 learned_addr learned my addr 172.21.15.47:0/2396844451 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:12.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.073+0000 7f1f8f7fe700 1 -- 172.21.15.47:0/2396844451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f90109820 con 0x7f1f90108670 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.073+0000 7f1f8f7fe700 1 --2- 172.21.15.47:0/2396844451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f90108a90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f800098a0 tx=0x7f1f80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e00886511f4d546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.073+0000 7f1f8e7fc700 1 -- 172.21.15.47:0/2396844451 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f80004160 con 0x7f1f90108670 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.073+0000 7f1f8e7fc700 1 -- 172.21.15.47:0/2396844451 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f80004300 con 0x7f1f90108670 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.073+0000 7f1f8e7fc700 1 -- 172.21.15.47:0/2396844451 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f80011af0 con 0x7f1f90108670 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.074+0000 7f1f96047700 1 -- 172.21.15.47:0/2396844451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 msgr2=0x7f1f90108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.074+0000 7f1f96047700 1 --2- 172.21.15.47:0/2396844451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f90108a90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f800098a0 tx=0x7f1f80009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.074+0000 7f1f96047700 1 -- 172.21.15.47:0/2396844451 shutdown_connections 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.074+0000 7f1f96047700 1 --2- 172.21.15.47:0/2396844451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f90108a90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.074+0000 7f1f96047700 1 -- 172.21.15.47:0/2396844451 >> 172.21.15.47:0/2396844451 conn(0x7f1f90103c30 msgr2=0x7f1f90106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.074+0000 7f1f96047700 1 -- 172.21.15.47:0/2396844451 shutdown_connections 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.074+0000 7f1f96047700 1 -- 172.21.15.47:0/2396844451 wait complete. 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.074+0000 7f1f96047700 1 Processor -- start 2024-04-25T20:49:12.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f96047700 1 -- start start 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f96047700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f9019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f96047700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9007d0e0 con 0x7f1f90108670 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f8f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f9019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f8f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f9019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38474/0 (socket says 172.21.15.47:38474) 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f8f7fe700 1 -- 172.21.15.47:0/2347584698 learned_addr learned my addr 172.21.15.47:0/2347584698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f8f7fe700 1 -- 172.21.15.47:0/2347584698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9019a870 con 0x7f1f90108670 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f8f7fe700 1 --2- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f9019a330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009fb0 tx=0x7f1f80011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f8cff9700 1 -- 172.21.15.47:0/2347584698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f800036a0 con 0x7f1f90108670 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.075+0000 7f1f8cff9700 1 -- 172.21.15.47:0/2347584698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f8001a430 con 0x7f1f90108670 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.076+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9019ab00 con 0x7f1f90108670 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.076+0000 7f1f8cff9700 1 -- 172.21.15.47:0/2347584698 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f80019550 con 0x7f1f90108670 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.076+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9019d7f0 con 0x7f1f90108670 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.076+0000 7f1f8cff9700 1 -- 172.21.15.47:0/2347584698 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f1f80019850 con 0x7f1f90108670 2024-04-25T20:49:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.077+0000 7f1f8cff9700 1 --2- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f1f7803d920 0x7f1f7803fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.077+0000 7f1f8cff9700 1 -- 172.21.15.47:0/2347584698 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f8004d8b0 con 0x7f1f90108670 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.077+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f900640a0 con 0x7f1f90108670 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.077+0000 7f1f8effd700 1 --2- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f1f7803d920 0x7f1f7803fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.077+0000 7f1f8effd700 1 --2- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f1f7803d920 0x7f1f7803fde0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006fd0 tx=0x7f1f84006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.082+0000 7f1f8cff9700 1 -- 172.21.15.47:0/2347584698 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f80023c90 con 0x7f1f90108670 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.346+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f1f9002e1d0 con 0x7f1f90108670 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.790+0000 7f1f8cff9700 1 -- 172.21.15.47:0/2347584698 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f1f8004d2d0 con 0x7f1f90108670 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.792+0000 7f1f8cff9700 1 -- 172.21.15.47:0/2347584698 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f1f8002d960 con 0x7f1f90108670 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f1f7803d920 msgr2=0x7f1f7803fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 --2- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f1f7803d920 0x7f1f7803fde0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006fd0 tx=0x7f1f84006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 msgr2=0x7f1f9019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 --2- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f9019a330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009fb0 tx=0x7f1f80011940 comp rx=0 tx=0).stop 2024-04-25T20:49:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 shutdown_connections 2024-04-25T20:49:12.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 --2- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f1f7803d920 0x7f1f7803fde0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:12.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 --2- 172.21.15.47:0/2347584698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f90108670 0x7f1f9019a330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:12.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 >> 172.21.15.47:0/2347584698 conn(0x7f1f90103c30 msgr2=0x7f1f90104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:12.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 shutdown_connections 2024-04-25T20:49:12.421 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:11.794+0000 7f1f96047700 1 -- 172.21.15.47:0/2347584698 wait complete. 2024-04-25T20:49:12.696 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:12.605+0000 7fcb8f181200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:49:13.037 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:12 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2347584698' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T20:49:13.037 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:12 smithi047 ceph-mon[29195]: mgrmap e5: x(active, since 6s) 2024-04-25T20:49:13.037 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:12.695+0000 7fcb8f181200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:49:13.898 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:13 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2377754655' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T20:49:13.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:49:13.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T20:49:13.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:49:13.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T20:49:13.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T20:49:13.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:49:13.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.829+0000 7f9aa7d12700 1 Processor -- start 2024-04-25T20:49:13.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.830+0000 7f9aa7d12700 1 -- start start 2024-04-25T20:49:13.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.830+0000 7f9aa7d12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa0108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:13.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.830+0000 7f9aa7d12700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa0109050 con 0x7f9aa0108660 2024-04-25T20:49:13.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.830+0000 7f9aa5aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa0108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:13.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.830+0000 7f9aa5aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa0108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38514/0 (socket says 172.21.15.47:38514) 2024-04-25T20:49:13.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.830+0000 7f9aa5aae700 1 -- 172.21.15.47:0/3401890606 learned_addr learned my addr 172.21.15.47:0/3401890606 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:13.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.830+0000 7f9aa5aae700 1 -- 172.21.15.47:0/3401890606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa0109810 con 0x7f9aa0108660 2024-04-25T20:49:13.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa5aae700 1 --2- 172.21.15.47:0/3401890606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa0108a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0098a0 tx=0x7f9a9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7391779cd875276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:13.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa4aac700 1 -- 172.21.15.47:0/3401890606 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a9c004160 con 0x7f9aa0108660 2024-04-25T20:49:13.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa4aac700 1 -- 172.21.15.47:0/3401890606 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9a9c004300 con 0x7f9aa0108660 2024-04-25T20:49:13.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa4aac700 1 -- 172.21.15.47:0/3401890606 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a9c011af0 con 0x7f9aa0108660 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa7d12700 1 -- 172.21.15.47:0/3401890606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 msgr2=0x7f9aa0108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa7d12700 1 --2- 172.21.15.47:0/3401890606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa0108a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0098a0 tx=0x7f9a9c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa7d12700 1 -- 172.21.15.47:0/3401890606 shutdown_connections 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa7d12700 1 --2- 172.21.15.47:0/3401890606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa0108a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.831+0000 7f9aa7d12700 1 -- 172.21.15.47:0/3401890606 >> 172.21.15.47:0/3401890606 conn(0x7f9aa0103c40 msgr2=0x7f9aa0106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.832+0000 7f9aa7d12700 1 -- 172.21.15.47:0/3401890606 shutdown_connections 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.832+0000 7f9aa7d12700 1 -- 172.21.15.47:0/3401890606 wait complete. 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.832+0000 7f9aa7d12700 1 Processor -- start 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.832+0000 7f9aa7d12700 1 -- start start 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.832+0000 7f9aa7d12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.832+0000 7f9aa7d12700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa010ca40 con 0x7f9aa0108660 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9aa5aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9aa5aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38526/0 (socket says 172.21.15.47:38526) 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9aa5aae700 1 -- 172.21.15.47:0/2377754655 learned_addr learned my addr 172.21.15.47:0/2377754655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9aa5aae700 1 -- 172.21.15.47:0/2377754655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa019ec80 con 0x7f9aa0108660 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9aa5aae700 1 --2- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa019e740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009fb0 tx=0x7f9a9c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9a96ffd700 1 -- 172.21.15.47:0/2377754655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a9c0036a0 con 0x7f9aa0108660 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9a96ffd700 1 -- 172.21.15.47:0/2377754655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9a9c01a430 con 0x7f9aa0108660 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa019ef10 con 0x7f9aa0108660 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa01a1c00 con 0x7f9aa0108660 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.833+0000 7f9a96ffd700 1 -- 172.21.15.47:0/2377754655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a9c019550 con 0x7f9aa0108660 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.834+0000 7f9a96ffd700 1 -- 172.21.15.47:0/2377754655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f9a9c019800 con 0x7f9aa0108660 2024-04-25T20:49:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.834+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa010cbc0 con 0x7f9aa0108660 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.835+0000 7f9a96ffd700 1 --2- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f9a8c0395b0 0x7f9a8c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.835+0000 7f9a96ffd700 1 -- 172.21.15.47:0/2377754655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9a9c04d870 con 0x7f9aa0108660 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.835+0000 7f9aa52ad700 1 -- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f9a8c0395b0 msgr2=0x7f9a8c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1508582803 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.835+0000 7f9aa52ad700 1 --2- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f9a8c0395b0 0x7f9a8c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:12.840+0000 7f9a96ffd700 1 -- 172.21.15.47:0/2377754655 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a9c04d290 con 0x7f9aa0108660 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.036+0000 7f9aa52ad700 1 -- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f9a8c0395b0 msgr2=0x7f9a8c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1508582803 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.036+0000 7f9aa52ad700 1 --2- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f9a8c0395b0 0x7f9a8c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.146+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9aa01a2250 con 0x7f9aa0108660 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.146+0000 7f9a96ffd700 1 -- 172.21.15.47:0/2377754655 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f9a9c016dd0 con 0x7f9aa0108660 2024-04-25T20:49:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.150+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f9a8c0395b0 msgr2=0x7f9a8c03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.150+0000 7f9aa7d12700 1 --2- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f9a8c0395b0 0x7f9a8c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.150+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 msgr2=0x7f9aa019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.150+0000 7f9aa7d12700 1 --2- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa019e740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009fb0 tx=0x7f9a9c011940 comp rx=0 tx=0).stop 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.151+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 shutdown_connections 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.151+0000 7f9aa7d12700 1 --2- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f9a8c0395b0 0x7f9a8c03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.151+0000 7f9aa7d12700 1 --2- 172.21.15.47:0/2377754655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9aa0108660 0x7f9aa019e740 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.151+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 >> 172.21.15.47:0/2377754655 conn(0x7f9aa0103c40 msgr2=0x7f9aa01046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.151+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 shutdown_connections 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:13.151+0000 7f9aa7d12700 1 -- 172.21.15.47:0/2377754655 wait complete. 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-04-25T20:49:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 5... 2024-04-25T20:49:14.158 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:13 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:13.970+0000 7fcb8f181200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:49:14.159 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:14.157+0000 7fcb8f181200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:49:14.544 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:14.337+0000 7fcb8f181200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:49:14.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:14.641+0000 7fcb8f181200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:49:14.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:14.737+0000 7fcb8f181200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:49:15.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:14.947+0000 7fcb8f181200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:49:15.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:15.149+0000 7fcb8f181200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:49:16.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:15.779+0000 7fcb8f181200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:49:16.529 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:16.218+0000 7fcb8f181200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:49:16.529 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:16.318+0000 7fcb8f181200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:49:16.530 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:16.528+0000 7fcb8f181200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:49:17.347 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:17.075+0000 7fcb8f181200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:49:17.348 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:17.171+0000 7fcb8f181200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:49:17.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:17.424+0000 7fcb8f181200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:49:17.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:17.564+0000 7fcb8f181200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:49:18.091 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:17.663+0000 7fcb8f181200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:49:18.091 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:17.783+0000 7fcb8f181200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:49:18.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:18.096+0000 7fcb8f181200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:49:18.959 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:18.732+0000 7fcb8f181200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:49:18.959 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:18.958+0000 7fcb8f181200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:49:19.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: Active manager daemon x restarted 2024-04-25T20:49:19.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: Activating manager daemon x 2024-04-25T20:49:19.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T20:49:19.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: mgrmap e6: x(active, starting, since 0.00321389s) 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:19.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:19 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:49:20.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:20 smithi047 ceph-mon[29195]: Found migration_current of "None". Setting to last migration. 2024-04-25T20:49:20.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:20 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:49:20.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:20 smithi047 ceph-mon[29195]: mgrmap e7: x(active, since 1.00579s) 2024-04-25T20:49:20.525 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:49:20.525 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T20:49:20.525 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.337+0000 7f7b2971b700 1 Processor -- start 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.337+0000 7f7b2971b700 1 -- start start 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.337+0000 7f7b2971b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b24107320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.337+0000 7f7b2971b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24063d00 con 0x7f7b24106f00 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.338+0000 7f7b22ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b24107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.338+0000 7f7b22ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b24107320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46962/0 (socket says 172.21.15.47:46962) 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.338+0000 7f7b22ffd700 1 -- 172.21.15.47:0/3665722810 learned_addr learned my addr 172.21.15.47:0/3665722810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.338+0000 7f7b22ffd700 1 -- 172.21.15.47:0/3665722810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b24107860 con 0x7f7b24106f00 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.338+0000 7f7b22ffd700 1 --2- 172.21.15.47:0/3665722810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b24107320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0098a0 tx=0x7f7b0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c44337ef212348a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.338+0000 7f7b21ffb700 1 -- 172.21.15.47:0/3665722810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b0c004160 con 0x7f7b24106f00 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.338+0000 7f7b21ffb700 1 -- 172.21.15.47:0/3665722810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b0c004300 con 0x7f7b24106f00 2024-04-25T20:49:20.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.338+0000 7f7b21ffb700 1 -- 172.21.15.47:0/3665722810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b0c011af0 con 0x7f7b24106f00 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.339+0000 7f7b2971b700 1 -- 172.21.15.47:0/3665722810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 msgr2=0x7f7b24107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.339+0000 7f7b2971b700 1 --2- 172.21.15.47:0/3665722810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b24107320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0098a0 tx=0x7f7b0c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.339+0000 7f7b2971b700 1 -- 172.21.15.47:0/3665722810 shutdown_connections 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.339+0000 7f7b2971b700 1 --2- 172.21.15.47:0/3665722810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b24107320 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.339+0000 7f7b2971b700 1 -- 172.21.15.47:0/3665722810 >> 172.21.15.47:0/3665722810 conn(0x7f7b24102b80 msgr2=0x7f7b24104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.339+0000 7f7b2971b700 1 -- 172.21.15.47:0/3665722810 shutdown_connections 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.339+0000 7f7b2971b700 1 -- 172.21.15.47:0/3665722810 wait complete. 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.340+0000 7f7b2971b700 1 Processor -- start 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.340+0000 7f7b2971b700 1 -- start start 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.340+0000 7f7b2971b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b2419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.340+0000 7f7b2971b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24108820 con 0x7f7b24106f00 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.340+0000 7f7b22ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b2419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:20.527 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.340+0000 7f7b22ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b2419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46972/0 (socket says 172.21.15.47:46972) 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.340+0000 7f7b22ffd700 1 -- 172.21.15.47:0/1274859547 learned_addr learned my addr 172.21.15.47:0/1274859547 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.340+0000 7f7b22ffd700 1 -- 172.21.15.47:0/1274859547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2419ec70 con 0x7f7b24106f00 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.341+0000 7f7b22ffd700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b2419e730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c011ac0 tx=0x7f7b0c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.341+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b0c003970 con 0x7f7b24106f00 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.341+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b0c003b10 con 0x7f7b24106f00 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.341+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2419ef00 con 0x7f7b24106f00 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.341+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b0c024610 con 0x7f7b24106f00 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.341+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b241a1bf0 con 0x7f7b24106f00 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.342+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7b0c024890 con 0x7f7b24106f00 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.342+0000 7f7b1bfff700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 0x7f7b1003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.342+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 --> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b240640a0 con 0x7f7b100395b0 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.342+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b0c04e8a0 con 0x7f7b24106f00 2024-04-25T20:49:20.528 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.342+0000 7f7b227fc700 1 -- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 msgr2=0x7f7b1003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1508582803 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.342+0000 7f7b227fc700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 0x7f7b1003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.543+0000 7f7b227fc700 1 -- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 msgr2=0x7f7b1003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1508582803 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.543+0000 7f7b227fc700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 0x7f7b1003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.944+0000 7f7b227fc700 1 -- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 msgr2=0x7f7b1003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1508582803 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:14.944+0000 7f7b227fc700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 0x7f7b1003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:15.745+0000 7f7b227fc700 1 -- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 msgr2=0x7f7b1003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1508582803 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:15.745+0000 7f7b227fc700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 0x7f7b1003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:17.346+0000 7f7b227fc700 1 -- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 msgr2=0x7f7b1003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1508582803 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:17.346+0000 7f7b227fc700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 0x7f7b1003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:18.961+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f7b0c02d910 con 0x7f7b24106f00 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:18.961+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 msgr2=0x7f7b1003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:18.961+0000 7f7b1bfff700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 0x7f7b1003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:20.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.962+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f7b0c02c870 con 0x7f7b24106f00 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.963+0000 7f7b1bfff700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7b1003ccb0 0x7f7b1003f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.963+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b0c03d0e0 con 0x7f7b1003ccb0 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.963+0000 7f7b227fc700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7b1003ccb0 0x7f7b1003f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.964+0000 7f7b227fc700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7b1003ccb0 0x7f7b1003f0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b14003a10 tx=0x7f7b14009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.965+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mgr.14120 v2:172.21.15.47:6800/2212759557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7b0c03d0e0 con 0x7f7b1003ccb0 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.971+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7b240754c0 con 0x7f7b1003ccb0 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.971+0000 7f7b1bfff700 1 -- 172.21.15.47:0/1274859547 <== mgr.14120 v2:172.21.15.47:6800/2212759557 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7b240754c0 con 0x7f7b1003ccb0 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7b1003ccb0 msgr2=0x7f7b1003f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:20.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7b1003ccb0 0x7f7b1003f0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b14003a10 tx=0x7f7b14009290 comp rx=0 tx=0).stop 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 msgr2=0x7f7b2419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b2419e730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c011ac0 tx=0x7f7b0c011ba0 comp rx=0 tx=0).stop 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 shutdown_connections 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7b1003ccb0 0x7f7b1003f0a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b24106f00 0x7f7b2419e730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 --2- 172.21.15.47:0/1274859547 >> [v2:172.21.15.47:6800/1508582803,v1:172.21.15.47:6801/1508582803] conn(0x7f7b100395b0 0x7f7b1003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 >> 172.21.15.47:0/1274859547 conn(0x7f7b24102b80 msgr2=0x7f7b24195310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 shutdown_connections 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:19.972+0000 7f7b2971b700 1 -- 172.21.15.47:0/1274859547 wait complete. 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 5 is available 2024-04-25T20:49:20.531 INFO:teuthology.orchestra.run.smithi047.stdout:Setting orchestrator backend to cephadm... 2024-04-25T20:49:21.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:21 smithi047 ceph-mon[29195]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T20:49:21.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:21 smithi047 ceph-mon[29195]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T20:49:21.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:21 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:21.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:21 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:21.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:21 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:21.693 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.932+0000 7f195f417700 1 Processor -- start 2024-04-25T20:49:21.693 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.933+0000 7f195f417700 1 -- start start 2024-04-25T20:49:21.693 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.933+0000 7f195f417700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f1958108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:21.693 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.933+0000 7f195f417700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1958109050 con 0x7f1958108660 2024-04-25T20:49:21.693 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.933+0000 7f195d1b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f1958108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:21.693 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.933+0000 7f195d1b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f1958108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47038/0 (socket says 172.21.15.47:47038) 2024-04-25T20:49:21.693 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.933+0000 7f195d1b3700 1 -- 172.21.15.47:0/878161250 learned_addr learned my addr 172.21.15.47:0/878161250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.933+0000 7f195d1b3700 1 -- 172.21.15.47:0/878161250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1958109810 con 0x7f1958108660 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.933+0000 7f195d1b3700 1 --2- 172.21.15.47:0/878161250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f1958108a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f19540098a0 tx=0x7f1954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab06519aab9862c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f194bfff700 1 -- 172.21.15.47:0/878161250 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954004160 con 0x7f1958108660 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f194bfff700 1 -- 172.21.15.47:0/878161250 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1954004300 con 0x7f1958108660 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f194bfff700 1 -- 172.21.15.47:0/878161250 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954011af0 con 0x7f1958108660 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f195f417700 1 -- 172.21.15.47:0/878161250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 msgr2=0x7f1958108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f195f417700 1 --2- 172.21.15.47:0/878161250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f1958108a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f19540098a0 tx=0x7f1954009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f195f417700 1 -- 172.21.15.47:0/878161250 shutdown_connections 2024-04-25T20:49:21.694 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f195f417700 1 --2- 172.21.15.47:0/878161250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f1958108a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f195f417700 1 -- 172.21.15.47:0/878161250 >> 172.21.15.47:0/878161250 conn(0x7f1958103c40 msgr2=0x7f1958106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f195f417700 1 -- 172.21.15.47:0/878161250 shutdown_connections 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.934+0000 7f195f417700 1 -- 172.21.15.47:0/878161250 wait complete. 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.935+0000 7f195f417700 1 Processor -- start 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.935+0000 7f195f417700 1 -- start start 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.935+0000 7f195f417700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f195819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.935+0000 7f195f417700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f195810ca40 con 0x7f1958108660 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.935+0000 7f195d1b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f195819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.935+0000 7f195d1b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f195819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47052/0 (socket says 172.21.15.47:47052) 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.935+0000 7f195d1b3700 1 -- 172.21.15.47:0/4104621420 learned_addr learned my addr 172.21.15.47:0/4104621420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.936+0000 7f195d1b3700 1 -- 172.21.15.47:0/4104621420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195819ca80 con 0x7f1958108660 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.936+0000 7f195d1b3700 1 --2- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f195819c540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19540117b0 tx=0x7f1954011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.936+0000 7f194a7fc700 1 -- 172.21.15.47:0/4104621420 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954003af0 con 0x7f1958108660 2024-04-25T20:49:21.695 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.936+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195819cd10 con 0x7f1958108660 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.936+0000 7f194a7fc700 1 -- 172.21.15.47:0/4104621420 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1954020070 con 0x7f1958108660 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.936+0000 7f194a7fc700 1 -- 172.21.15.47:0/4104621420 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954019630 con 0x7f1958108660 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.936+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195819fa00 con 0x7f1958108660 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.937+0000 7f194a7fc700 1 -- 172.21.15.47:0/4104621420 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f19540198f0 con 0x7f1958108660 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.937+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193c005360 con 0x7f1958108660 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.937+0000 7f194a7fc700 1 --2- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f1944039490 0x7f194403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.937+0000 7f194a7fc700 1 -- 172.21.15.47:0/4104621420 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f195404da80 con 0x7f1958108660 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.937+0000 7f195c9b2700 1 --2- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f1944039490 0x7f194403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.938+0000 7f195c9b2700 1 --2- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f1944039490 0x7f194403b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f194c006fd0 tx=0x7f194c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:20.942+0000 7f194a7fc700 1 -- 172.21.15.47:0/4104621420 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f195402dc70 con 0x7f1958108660 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.168+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f193c000bf0 con 0x7f1944039490 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.174+0000 7f194a7fc700 1 -- 172.21.15.47:0/4104621420 <== mgr.14120 v2:172.21.15.47:6800/2212759557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f193c000bf0 con 0x7f1944039490 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.176+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f1944039490 msgr2=0x7f194403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.176+0000 7f195f417700 1 --2- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f1944039490 0x7f194403b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f194c006fd0 tx=0x7f194c006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.177+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 msgr2=0x7f195819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:21.697 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.177+0000 7f195f417700 1 --2- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f195819c540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19540117b0 tx=0x7f1954011b70 comp rx=0 tx=0).stop 2024-04-25T20:49:21.697 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.177+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 shutdown_connections 2024-04-25T20:49:21.697 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.177+0000 7f195f417700 1 --2- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f1944039490 0x7f194403b950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:21.697 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.177+0000 7f195f417700 1 --2- 172.21.15.47:0/4104621420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1958108660 0x7f195819c540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:21.697 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.177+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 >> 172.21.15.47:0/4104621420 conn(0x7f1958103c40 msgr2=0x7f19581047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:21.697 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.177+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 shutdown_connections 2024-04-25T20:49:21.697 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:21.177+0000 7f195f417700 1 -- 172.21.15.47:0/4104621420 wait complete. 2024-04-25T20:49:22.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:22 smithi047 ceph-mon[29195]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:22.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:22 smithi047 ceph-mon[29195]: mgrmap e8: x(active, since 2s) 2024-04-25T20:49:22.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:22 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:22.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:22 smithi047 ceph-mon[29195]: [25/Apr/2024:20:49:21] ENGINE Bus STARTING 2024-04-25T20:49:22.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:22 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.123+0000 7f4e5710d700 1 Processor -- start 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.123+0000 7f4e5710d700 1 -- start start 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.123+0000 7f4e5710d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e50107330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.123+0000 7f4e5710d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5007ce30 con 0x7f4e50106f10 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.123+0000 7f4e54ea9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e50107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.123+0000 7f4e54ea9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e50107330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47066/0 (socket says 172.21.15.47:47066) 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.123+0000 7f4e54ea9700 1 -- 172.21.15.47:0/4115795284 learned_addr learned my addr 172.21.15.47:0/4115795284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.124+0000 7f4e54ea9700 1 -- 172.21.15.47:0/4115795284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e50107870 con 0x7f4e50106f10 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.124+0000 7f4e54ea9700 1 --2- 172.21.15.47:0/4115795284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e50107330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e440098a0 tx=0x7f4e44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47e654eda637c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.124+0000 7f4e4f7fe700 1 -- 172.21.15.47:0/4115795284 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e44004160 con 0x7f4e50106f10 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.124+0000 7f4e4f7fe700 1 -- 172.21.15.47:0/4115795284 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e44004300 con 0x7f4e50106f10 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.124+0000 7f4e4f7fe700 1 -- 172.21.15.47:0/4115795284 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e44011af0 con 0x7f4e50106f10 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.124+0000 7f4e5710d700 1 -- 172.21.15.47:0/4115795284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 msgr2=0x7f4e50107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.124+0000 7f4e5710d700 1 --2- 172.21.15.47:0/4115795284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e50107330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e440098a0 tx=0x7f4e44009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.124+0000 7f4e5710d700 1 -- 172.21.15.47:0/4115795284 shutdown_connections 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.125+0000 7f4e5710d700 1 --2- 172.21.15.47:0/4115795284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e50107330 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.125+0000 7f4e5710d700 1 -- 172.21.15.47:0/4115795284 >> 172.21.15.47:0/4115795284 conn(0x7f4e50102b70 msgr2=0x7f4e50104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.125+0000 7f4e5710d700 1 -- 172.21.15.47:0/4115795284 shutdown_connections 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.125+0000 7f4e5710d700 1 -- 172.21.15.47:0/4115795284 wait complete. 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.125+0000 7f4e5710d700 1 Processor -- start 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.125+0000 7f4e5710d700 1 -- start start 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.125+0000 7f4e5710d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e5019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:22.927 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.125+0000 7f4e5710d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50108830 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e54ea9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e5019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e54ea9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e5019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47076/0 (socket says 172.21.15.47:47076) 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e54ea9700 1 -- 172.21.15.47:0/834946622 learned_addr learned my addr 172.21.15.47:0/834946622 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e54ea9700 1 -- 172.21.15.47:0/834946622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5019ec50 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e54ea9700 1 --2- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e5019e710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009fb0 tx=0x7f4e44011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e4dffb700 1 -- 172.21.15.47:0/834946622 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e440036a0 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e4dffb700 1 -- 172.21.15.47:0/834946622 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e4401a430 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5019eee0 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e4dffb700 1 -- 172.21.15.47:0/834946622 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e44019550 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.126+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501a1bd0 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.127+0000 7f4e4dffb700 1 -- 172.21.15.47:0/834946622 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4e44019800 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.127+0000 7f4e4dffb700 1 --2- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f4e400395b0 0x7f4e4003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.127+0000 7f4e4dffb700 1 -- 172.21.15.47:0/834946622 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e4404d8a0 con 0x7f4e50106f10 2024-04-25T20:49:22.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.127+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e500640a0 con 0x7f4e50106f10 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.127+0000 7f4e4ffff700 1 --2- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f4e400395b0 0x7f4e4003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.128+0000 7f4e4ffff700 1 --2- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f4e400395b0 0x7f4e4003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c006fd0 tx=0x7f4e3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.133+0000 7f4e4dffb700 1 -- 172.21.15.47:0/834946622 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e44023c90 con 0x7f4e50106f10 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.343+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f4e5002e200 con 0x7f4e400395b0 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.344+0000 7f4e4dffb700 1 -- 172.21.15.47:0/834946622 <== mgr.14120 v2:172.21.15.47:6800/2212759557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f4e5002e200 con 0x7f4e400395b0 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f4e400395b0 msgr2=0x7f4e4003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 --2- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f4e400395b0 0x7f4e4003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c006fd0 tx=0x7f4e3c006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 msgr2=0x7f4e5019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 --2- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e5019e710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009fb0 tx=0x7f4e44011940 comp rx=0 tx=0).stop 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 shutdown_connections 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 --2- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f4e400395b0 0x7f4e4003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 --2- 172.21.15.47:0/834946622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50106f10 0x7f4e5019e710 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 >> 172.21.15.47:0/834946622 conn(0x7f4e50102b70 msgr2=0x7f4e50195330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 shutdown_connections 2024-04-25T20:49:22.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:22.346+0000 7f4e5710d700 1 -- 172.21.15.47:0/834946622 wait complete. 2024-04-25T20:49:22.930 INFO:teuthology.orchestra.run.smithi047.stdout:Generating ssh key... 2024-04-25T20:49:23.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-mon[29195]: [25/Apr/2024:20:49:21] ENGINE Serving on https://172.21.15.47:7150 2024-04-25T20:49:23.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-mon[29195]: [25/Apr/2024:20:49:22] ENGINE Serving on http://172.21.15.47:8765 2024-04-25T20:49:23.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-mon[29195]: [25/Apr/2024:20:49:22] ENGINE Bus STARTED 2024-04-25T20:49:23.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-mon[29195]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:24.298 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:24 smithi047 ceph-mon[29195]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:24.298 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:24 smithi047 ceph-mon[29195]: Generating ssh key... 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:24 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:24 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: Generating public/private rsa key pair. 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: Your identification has been saved in /tmp/tmp6s28al3h/key. 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: Your public key has been saved in /tmp/tmp6s28al3h/key.pub. 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: The key fingerprint is: 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: SHA256:3ouF2n0MW7nKYD/zki07ZcHhi9gSFo8Ea/+0C4pV65k ceph-0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: The key's randomart image is: 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: +---[RSA 3072]----+ 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | .. | 2024-04-25T20:49:24.299 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | .o . | 2024-04-25T20:49:24.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | o. + o . | 2024-04-25T20:49:24.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | . .+ . + | 2024-04-25T20:49:24.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | .S+.. + | 2024-04-25T20:49:24.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | oo*+.* | 2024-04-25T20:49:24.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | . B.=X . | 2024-04-25T20:49:24.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | o * @Oo= | 2024-04-25T20:49:24.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: | . o E O@. | 2024-04-25T20:49:24.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: +----[SHA256]-----+ 2024-04-25T20:49:24.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.348+0000 7fb42ba63700 1 Processor -- start 2024-04-25T20:49:24.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.348+0000 7fb42ba63700 1 -- start start 2024-04-25T20:49:24.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb42ba63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb424108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:24.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb42ba63700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb424109060 con 0x7fb424108670 2024-04-25T20:49:24.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb4297ff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb424108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:24.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb4297ff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb424108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47084/0 (socket says 172.21.15.47:47084) 2024-04-25T20:49:24.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb4297ff700 1 -- 172.21.15.47:0/935301122 learned_addr learned my addr 172.21.15.47:0/935301122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb4297ff700 1 -- 172.21.15.47:0/935301122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb424109820 con 0x7fb424108670 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb4297ff700 1 --2- 172.21.15.47:0/935301122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb424108a90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4200098a0 tx=0x7fb420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fe2391b5ab78b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb41bfff700 1 -- 172.21.15.47:0/935301122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb420004160 con 0x7fb424108670 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb41bfff700 1 -- 172.21.15.47:0/935301122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb420004300 con 0x7fb424108670 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.349+0000 7fb41bfff700 1 -- 172.21.15.47:0/935301122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb420011af0 con 0x7fb424108670 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.350+0000 7fb42ba63700 1 -- 172.21.15.47:0/935301122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 msgr2=0x7fb424108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.350+0000 7fb42ba63700 1 --2- 172.21.15.47:0/935301122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb424108a90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4200098a0 tx=0x7fb420009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.350+0000 7fb42ba63700 1 -- 172.21.15.47:0/935301122 shutdown_connections 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.350+0000 7fb42ba63700 1 --2- 172.21.15.47:0/935301122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb424108a90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.350+0000 7fb42ba63700 1 -- 172.21.15.47:0/935301122 >> 172.21.15.47:0/935301122 conn(0x7fb424103c30 msgr2=0x7fb424106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.350+0000 7fb42ba63700 1 -- 172.21.15.47:0/935301122 shutdown_connections 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.350+0000 7fb42ba63700 1 -- 172.21.15.47:0/935301122 wait complete. 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.351+0000 7fb42ba63700 1 Processor -- start 2024-04-25T20:49:24.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.351+0000 7fb42ba63700 1 -- start start 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.351+0000 7fb42ba63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb42419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.351+0000 7fb42ba63700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb42410ca50 con 0x7fb424108670 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.351+0000 7fb4297ff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb42419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.351+0000 7fb4297ff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb42419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47092/0 (socket says 172.21.15.47:47092) 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.351+0000 7fb4297ff700 1 -- 172.21.15.47:0/2944994896 learned_addr learned my addr 172.21.15.47:0/2944994896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.352+0000 7fb4297ff700 1 -- 172.21.15.47:0/2944994896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42419ca90 con 0x7fb424108670 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.352+0000 7fb4297ff700 1 --2- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb42419c550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4200117b0 tx=0x7fb420011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.352+0000 7fb41a7fc700 1 -- 172.21.15.47:0/2944994896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb420003af0 con 0x7fb424108670 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.352+0000 7fb41a7fc700 1 -- 172.21.15.47:0/2944994896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb420020070 con 0x7fb424108670 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.352+0000 7fb41a7fc700 1 -- 172.21.15.47:0/2944994896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb420019630 con 0x7fb424108670 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.352+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42419cd20 con 0x7fb424108670 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.352+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42419fa10 con 0x7fb424108670 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.353+0000 7fb41a7fc700 1 -- 172.21.15.47:0/2944994896 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb4200198a0 con 0x7fb424108670 2024-04-25T20:49:24.577 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.353+0000 7fb41a7fc700 1 --2- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fb410039600 0x7fb41003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.353+0000 7fb41a7fc700 1 -- 172.21.15.47:0/2944994896 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb42004db60 con 0x7fb424108670 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.353+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42410cbd0 con 0x7fb424108670 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.353+0000 7fb428ffe700 1 --2- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fb410039600 0x7fb41003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.353+0000 7fb428ffe700 1 --2- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fb410039600 0x7fb41003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb414006fd0 tx=0x7fb414006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.358+0000 7fb41a7fc700 1 -- 172.21.15.47:0/2944994896 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb420023770 con 0x7fb424108670 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.567+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fb424104e80 con 0x7fb410039600 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.981+0000 7fb41a7fc700 1 -- 172.21.15.47:0/2944994896 <== mgr.14120 v2:172.21.15.47:6800/2212759557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb424104e80 con 0x7fb410039600 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.983+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fb410039600 msgr2=0x7fb41003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.983+0000 7fb42ba63700 1 --2- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fb410039600 0x7fb41003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb414006fd0 tx=0x7fb414006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.983+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 msgr2=0x7fb42419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.983+0000 7fb42ba63700 1 --2- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb42419c550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4200117b0 tx=0x7fb420011b70 comp rx=0 tx=0).stop 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.983+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 shutdown_connections 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.983+0000 7fb42ba63700 1 --2- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fb410039600 0x7fb41003bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:24.578 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.983+0000 7fb42ba63700 1 --2- 172.21.15.47:0/2944994896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424108670 0x7fb42419c550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:24.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.983+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 >> 172.21.15.47:0/2944994896 conn(0x7fb424103c30 msgr2=0x7fb424104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:24.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.984+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 shutdown_connections 2024-04-25T20:49:24.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:23.984+0000 7fb42ba63700 1 -- 172.21.15.47:0/2944994896 wait complete. 2024-04-25T20:49:25.713 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9Njub0SmJWpO26SeOpG/cUp4ezSVV+Yo7UTuMz/Z4Xk/EBMitfnBO+NMiuLQG+TRA3SBkhtup3f+5lNcQGhDqeX4qSPGFVRf6VVH7i3iRKa5tyqYnTONlOn2T015+q1IaccIsYiVXIyaBQttz3A/zrKoaJwkxSoptXP+iU/WHzai+GqUYmYo6acUKrWkR8oW6vVrgQ16c0UWP8g69kX1pDzHuJl9s1I8N0O08tMLatYhkVrBGfwaAmdb0DWjYnXGIp5Yfp0JKoiiZMDa00ZE/Fw+yqyTpVV5ww3tpAu2dw/pF6TDMOAmEVN+8wUTzNsOc4ZXWTYAGruwn4CUXATlmlPwRJWnkXgrkQu2jKRFgVJfYndrZK+OX9YMpEpkVlAFOJeuv4uJGeLBZzVA93uScBKsM2Gqre2nkseAUKJNjk3S31p/kNAkSFa7yTm6FiQpkhFjjLuVDPupd1m1QOJNvz0PklHKqExUv+dudXkiGNrxHmqzM0kVo9wv8Wnp4wnE= ceph-0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:49:25.713 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.982+0000 7f6aec305700 1 Processor -- start 2024-04-25T20:49:25.713 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.982+0000 7f6aec305700 1 -- start start 2024-04-25T20:49:25.713 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.983+0000 7f6aec305700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae4108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:25.713 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.983+0000 7f6aec305700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4109060 con 0x7f6ae4108670 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.983+0000 7f6aea0a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae4108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.983+0000 7f6aea0a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae4108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32838/0 (socket says 172.21.15.47:32838) 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.983+0000 7f6aea0a1700 1 -- 172.21.15.47:0/2083137136 learned_addr learned my addr 172.21.15.47:0/2083137136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.983+0000 7f6aea0a1700 1 -- 172.21.15.47:0/2083137136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae4109820 con 0x7f6ae4108670 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.983+0000 7f6aea0a1700 1 --2- 172.21.15.47:0/2083137136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae4108a90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40098a0 tx=0x7f6ad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614789b4ba26d7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6ae909f700 1 -- 172.21.15.47:0/2083137136 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4004160 con 0x7f6ae4108670 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6ae909f700 1 -- 172.21.15.47:0/2083137136 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ad4004300 con 0x7f6ae4108670 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6ae909f700 1 -- 172.21.15.47:0/2083137136 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4011af0 con 0x7f6ae4108670 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6aec305700 1 -- 172.21.15.47:0/2083137136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 msgr2=0x7f6ae4108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6aec305700 1 --2- 172.21.15.47:0/2083137136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae4108a90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40098a0 tx=0x7f6ad4009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6aec305700 1 -- 172.21.15.47:0/2083137136 shutdown_connections 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6aec305700 1 --2- 172.21.15.47:0/2083137136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae4108a90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6aec305700 1 -- 172.21.15.47:0/2083137136 >> 172.21.15.47:0/2083137136 conn(0x7f6ae4103c30 msgr2=0x7f6ae4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6aec305700 1 -- 172.21.15.47:0/2083137136 shutdown_connections 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.984+0000 7f6aec305700 1 -- 172.21.15.47:0/2083137136 wait complete. 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.985+0000 7f6aec305700 1 Processor -- start 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.985+0000 7f6aec305700 1 -- start start 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.985+0000 7f6aec305700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.985+0000 7f6aec305700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae410ca50 con 0x7f6ae4108670 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.985+0000 7f6aea0a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.985+0000 7f6aea0a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32850/0 (socket says 172.21.15.47:32850) 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.985+0000 7f6aea0a1700 1 -- 172.21.15.47:0/574807355 learned_addr learned my addr 172.21.15.47:0/574807355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.986+0000 7f6aea0a1700 1 -- 172.21.15.47:0/574807355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae419ca90 con 0x7f6ae4108670 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.986+0000 7f6aea0a1700 1 --2- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae419c550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009fb0 tx=0x7f6ad4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.986+0000 7f6adb7fe700 1 -- 172.21.15.47:0/574807355 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad40036a0 con 0x7f6ae4108670 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.986+0000 7f6adb7fe700 1 -- 172.21.15.47:0/574807355 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ad401a430 con 0x7f6ae4108670 2024-04-25T20:49:25.715 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.986+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae419cd20 con 0x7f6ae4108670 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.986+0000 7f6adb7fe700 1 -- 172.21.15.47:0/574807355 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4019550 con 0x7f6ae4108670 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.986+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae419fa10 con 0x7f6ae4108670 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.987+0000 7f6adb7fe700 1 -- 172.21.15.47:0/574807355 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6ad4019800 con 0x7f6ae4108670 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.987+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae410cbd0 con 0x7f6ae4108670 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.987+0000 7f6adb7fe700 1 --2- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6ad00395b0 0x7f6ad003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.987+0000 7f6adb7fe700 1 -- 172.21.15.47:0/574807355 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ad404da10 con 0x7f6ae4108670 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.987+0000 7f6ae98a0700 1 --2- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6ad00395b0 0x7f6ad003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.988+0000 7f6ae98a0700 1 --2- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6ad00395b0 0x7f6ad003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0006fd0 tx=0x7f6ae0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:24.992+0000 7f6adb7fe700 1 -- 172.21.15.47:0/574807355 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ad402d960 con 0x7f6ae4108670 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.202+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f6ae4104e80 con 0x7f6ad00395b0 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.203+0000 7f6adb7fe700 1 -- 172.21.15.47:0/574807355 <== mgr.14120 v2:172.21.15.47:6800/2212759557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f6ae4104e80 con 0x7f6ad00395b0 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6ad00395b0 msgr2=0x7f6ad003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 --2- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6ad00395b0 0x7f6ad003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0006fd0 tx=0x7f6ae0006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 msgr2=0x7f6ae419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:25.716 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 --2- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae419c550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009fb0 tx=0x7f6ad4011940 comp rx=0 tx=0).stop 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 shutdown_connections 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 --2- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6ad00395b0 0x7f6ad003ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 --2- 172.21.15.47:0/574807355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae4108670 0x7f6ae419c550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 >> 172.21.15.47:0/574807355 conn(0x7f6ae4103c30 msgr2=0x7f6ae4104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 shutdown_connections 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:25.205+0000 7f6aec305700 1 -- 172.21.15.47:0/574807355 wait complete. 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T20:49:25.717 INFO:teuthology.orchestra.run.smithi047.stdout:Adding host smithi047... 2024-04-25T20:49:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:25 smithi047 ceph-mon[29195]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:26.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:26 smithi047 ceph-mon[29195]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:28.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:27 smithi047 ceph-mon[29195]: Deploying cephadm binary to smithi047 2024-04-25T20:49:31.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Added host 'smithi047' with addr '172.21.15.47' 2024-04-25T20:49:31.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.138+0000 7f2570f28700 1 Processor -- start 2024-04-25T20:49:31.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.138+0000 7f2570f28700 1 -- start start 2024-04-25T20:49:31.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.138+0000 7f2570f28700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.138+0000 7f2570f28700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f256c07d8d0 con 0x7f256c07eea0 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.138+0000 7f256a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.139+0000 7f256a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32858/0 (socket says 172.21.15.47:32858) 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.139+0000 7f256a59c700 1 -- 172.21.15.47:0/1920091942 learned_addr learned my addr 172.21.15.47:0/1920091942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.139+0000 7f256a59c700 1 -- 172.21.15.47:0/1920091942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c07da50 con 0x7f256c07eea0 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.139+0000 7f256a59c700 1 --2- 172.21.15.47:0/1920091942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c07d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25640098a0 tx=0x7f2564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8193814bf2b67c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.139+0000 7f256959a700 1 -- 172.21.15.47:0/1920091942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2564004160 con 0x7f256c07eea0 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.139+0000 7f256959a700 1 -- 172.21.15.47:0/1920091942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2564004300 con 0x7f256c07eea0 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.139+0000 7f256959a700 1 -- 172.21.15.47:0/1920091942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2564011af0 con 0x7f256c07eea0 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.140+0000 7f2570f28700 1 -- 172.21.15.47:0/1920091942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 msgr2=0x7f256c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.140+0000 7f2570f28700 1 --2- 172.21.15.47:0/1920091942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c07d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25640098a0 tx=0x7f2564009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.140+0000 7f2570f28700 1 -- 172.21.15.47:0/1920091942 shutdown_connections 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.140+0000 7f2570f28700 1 --2- 172.21.15.47:0/1920091942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c07d300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:31.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.140+0000 7f2570f28700 1 -- 172.21.15.47:0/1920091942 >> 172.21.15.47:0/1920091942 conn(0x7f256c103ba0 msgr2=0x7f256c105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.140+0000 7f2570f28700 1 -- 172.21.15.47:0/1920091942 shutdown_connections 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.140+0000 7f2570f28700 1 -- 172.21.15.47:0/1920091942 wait complete. 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f2570f28700 1 Processor -- start 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f2570f28700 1 -- start start 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f2570f28700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f2570f28700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f256c10eb30 con 0x7f256c07eea0 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f256a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f256a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32870/0 (socket says 172.21.15.47:32870) 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f256a59c700 1 -- 172.21.15.47:0/3838275675 learned_addr learned my addr 172.21.15.47:0/3838275675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f256a59c700 1 -- 172.21.15.47:0/3838275675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c19ec90 con 0x7f256c07eea0 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f256a59c700 1 --2- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c19e750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25640117b0 tx=0x7f2564011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f25637fe700 1 -- 172.21.15.47:0/3838275675 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2564003af0 con 0x7f256c07eea0 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f25637fe700 1 -- 172.21.15.47:0/3838275675 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2564020070 con 0x7f256c07eea0 2024-04-25T20:49:31.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.141+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c19ef20 con 0x7f256c07eea0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.142+0000 7f25637fe700 1 -- 172.21.15.47:0/3838275675 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2564019630 con 0x7f256c07eea0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.142+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c1a1c10 con 0x7f256c07eea0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.142+0000 7f25637fe700 1 -- 172.21.15.47:0/3838275675 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f256401a9c0 con 0x7f256c07eea0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.143+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f256c0640a0 con 0x7f256c07eea0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.143+0000 7f25637fe700 1 --2- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f2558039600 0x7f255803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.143+0000 7f25637fe700 1 -- 172.21.15.47:0/3838275675 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f256404db30 con 0x7f256c07eea0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.143+0000 7f2569d9b700 1 --2- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f2558039600 0x7f255803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.143+0000 7f2569d9b700 1 --2- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f2558039600 0x7f255803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2554006fd0 tx=0x7f2554006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.148+0000 7f25637fe700 1 -- 172.21.15.47:0/3838275675 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f256400e260 con 0x7f256c07eea0 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:26.362+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}) -- 0x7f256c104e60 con 0x7f2558039600 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.659+0000 7f25637fe700 1 -- 172.21.15.47:0/3838275675 <== mgr.14120 v2:172.21.15.47:6800/2212759557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f256c104e60 con 0x7f2558039600 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f2558039600 msgr2=0x7f255803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:31.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 --2- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f2558039600 0x7f255803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2554006fd0 tx=0x7f2554006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 msgr2=0x7f256c19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 --2- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c19e750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25640117b0 tx=0x7f2564011b70 comp rx=0 tx=0).stop 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 shutdown_connections 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 --2- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f2558039600 0x7f255803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 --2- 172.21.15.47:0/3838275675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f256c07eea0 0x7f256c19e750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 >> 172.21.15.47:0/3838275675 conn(0x7f256c103ba0 msgr2=0x7f256c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 shutdown_connections 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:30.662+0000 7f2570f28700 1 -- 172.21.15.47:0/3838275675 wait complete. 2024-04-25T20:49:31.242 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mon service... 2024-04-25T20:49:31.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:31 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:31.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:31 smithi047 ceph-mon[29195]: Added host smithi047 2024-04-25T20:49:31.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:31 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:32.701 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T20:49:32.701 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.764+0000 7feaa8e42700 1 Processor -- start 2024-04-25T20:49:32.701 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.764+0000 7feaa8e42700 1 -- start start 2024-04-25T20:49:32.701 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.764+0000 7feaa8e42700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa4107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:32.701 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.764+0000 7feaa8e42700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4107f00 con 0x7feaa4107510 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.765+0000 7feaa259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa4107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.765+0000 7feaa259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa4107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32878/0 (socket says 172.21.15.47:32878) 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.765+0000 7feaa259c700 1 -- 172.21.15.47:0/924478909 learned_addr learned my addr 172.21.15.47:0/924478909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.765+0000 7feaa259c700 1 -- 172.21.15.47:0/924478909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa4108710 con 0x7feaa4107510 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.765+0000 7feaa259c700 1 --2- 172.21.15.47:0/924478909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa4107930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00d180 tx=0x7fea9c00d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116e2a3469716435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.765+0000 7feaa159a700 1 -- 172.21.15.47:0/924478909 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea9c019070 con 0x7feaa4107510 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.765+0000 7feaa159a700 1 -- 172.21.15.47:0/924478909 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fea9c00de20 con 0x7feaa4107510 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.765+0000 7feaa159a700 1 -- 172.21.15.47:0/924478909 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea9c009a70 con 0x7feaa4107510 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.766+0000 7feaa8e42700 1 -- 172.21.15.47:0/924478909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 msgr2=0x7feaa4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.766+0000 7feaa8e42700 1 --2- 172.21.15.47:0/924478909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa4107930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00d180 tx=0x7fea9c00d490 comp rx=0 tx=0).stop 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.766+0000 7feaa8e42700 1 -- 172.21.15.47:0/924478909 shutdown_connections 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.766+0000 7feaa8e42700 1 --2- 172.21.15.47:0/924478909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa4107930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.766+0000 7feaa8e42700 1 -- 172.21.15.47:0/924478909 >> 172.21.15.47:0/924478909 conn(0x7feaa4102b10 msgr2=0x7feaa4104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.766+0000 7feaa8e42700 1 -- 172.21.15.47:0/924478909 shutdown_connections 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.766+0000 7feaa8e42700 1 -- 172.21.15.47:0/924478909 wait complete. 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa8e42700 1 Processor -- start 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa8e42700 1 -- start start 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa8e42700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa8e42700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feaa407bfd0 con 0x7feaa4107510 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32892/0 (socket says 172.21.15.47:32892) 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa259c700 1 -- 172.21.15.47:0/829459460 learned_addr learned my addr 172.21.15.47:0/829459460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa259c700 1 -- 172.21.15.47:0/829459460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa419ec40 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.767+0000 7feaa259c700 1 --2- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa419e700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009640 tx=0x7fea9c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.768+0000 7fea9b7fe700 1 -- 172.21.15.47:0/829459460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea9c019040 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.768+0000 7fea9b7fe700 1 -- 172.21.15.47:0/829459460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fea9c003c10 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.768+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa419eed0 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.768+0000 7fea9b7fe700 1 -- 172.21.15.47:0/829459460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea9c01f7d0 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.768+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa41a1bc0 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.769+0000 7fea9b7fe700 1 -- 172.21.15.47:0/829459460 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fea9c01fa00 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.769+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa40640a0 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.769+0000 7fea9b7fe700 1 --2- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fea900395b0 0x7fea9003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.769+0000 7fea9b7fe700 1 -- 172.21.15.47:0/829459460 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea9c011050 con 0x7feaa4107510 2024-04-25T20:49:32.704 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.769+0000 7feaa1d9b700 1 --2- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fea900395b0 0x7fea9003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.769+0000 7feaa1d9b700 1 --2- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fea900395b0 0x7fea9003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea8c006fd0 tx=0x7fea8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.774+0000 7fea9b7fe700 1 -- 172.21.15.47:0/829459460 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea9c01fdf0 con 0x7feaa4107510 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:31.998+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7feaa40754c0 con 0x7fea900395b0 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.003+0000 7fea9b7fe700 1 -- 172.21.15.47:0/829459460 <== mgr.14120 v2:172.21.15.47:6800/2212759557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7feaa40754c0 con 0x7fea900395b0 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fea900395b0 msgr2=0x7fea9003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 --2- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fea900395b0 0x7fea9003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea8c006fd0 tx=0x7fea8c006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 msgr2=0x7feaa419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 --2- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa419e700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009640 tx=0x7fea9c009490 comp rx=0 tx=0).stop 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 shutdown_connections 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 --2- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fea900395b0 0x7fea9003ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 --2- 172.21.15.47:0/829459460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaa4107510 0x7feaa419e700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:32.705 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 >> 172.21.15.47:0/829459460 conn(0x7feaa4102b10 msgr2=0x7feaa4190930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:32.706 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 shutdown_connections 2024-04-25T20:49:32.706 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:32.005+0000 7feaa8e42700 1 -- 172.21.15.47:0/829459460 wait complete. 2024-04-25T20:49:32.706 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mgr service... 2024-04-25T20:49:33.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:33 smithi047 ceph-mon[29195]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:33.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:33 smithi047 ceph-mon[29195]: Saving service mon spec with placement count:5 2024-04-25T20:49:33.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:33 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.199+0000 7f7c97b0d700 1 Processor -- start 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.199+0000 7f7c97b0d700 1 -- start start 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.200+0000 7f7c97b0d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c90108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.200+0000 7f7c97b0d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90109050 con 0x7f7c90108660 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.200+0000 7f7c958a9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c90108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.200+0000 7f7c958a9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c90108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32902/0 (socket says 172.21.15.47:32902) 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.200+0000 7f7c958a9700 1 -- 172.21.15.47:0/1325506890 learned_addr learned my addr 172.21.15.47:0/1325506890 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.200+0000 7f7c958a9700 1 -- 172.21.15.47:0/1325506890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c90109810 con 0x7f7c90108660 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.200+0000 7f7c958a9700 1 --2- 172.21.15.47:0/1325506890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c90108a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0098a0 tx=0x7f7c8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64987b411b279338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:34.434 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.200+0000 7f7c948a7700 1 -- 172.21.15.47:0/1325506890 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c8c004160 con 0x7f7c90108660 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c948a7700 1 -- 172.21.15.47:0/1325506890 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7c8c004300 con 0x7f7c90108660 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c948a7700 1 -- 172.21.15.47:0/1325506890 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c8c011af0 con 0x7f7c90108660 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c97b0d700 1 -- 172.21.15.47:0/1325506890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 msgr2=0x7f7c90108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c97b0d700 1 --2- 172.21.15.47:0/1325506890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c90108a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0098a0 tx=0x7f7c8c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c97b0d700 1 -- 172.21.15.47:0/1325506890 shutdown_connections 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c97b0d700 1 --2- 172.21.15.47:0/1325506890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c90108a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c97b0d700 1 -- 172.21.15.47:0/1325506890 >> 172.21.15.47:0/1325506890 conn(0x7f7c90103c40 msgr2=0x7f7c90106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c97b0d700 1 -- 172.21.15.47:0/1325506890 shutdown_connections 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.201+0000 7f7c97b0d700 1 -- 172.21.15.47:0/1325506890 wait complete. 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.202+0000 7f7c97b0d700 1 Processor -- start 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.202+0000 7f7c97b0d700 1 -- start start 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.202+0000 7f7c97b0d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c9019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.202+0000 7f7c97b0d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9010ca40 con 0x7f7c90108660 2024-04-25T20:49:34.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.202+0000 7f7c958a9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c9019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.202+0000 7f7c958a9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c9019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32910/0 (socket says 172.21.15.47:32910) 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.202+0000 7f7c958a9700 1 -- 172.21.15.47:0/384347660 learned_addr learned my addr 172.21.15.47:0/384347660 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.203+0000 7f7c958a9700 1 -- 172.21.15.47:0/384347660 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9019ca80 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.203+0000 7f7c958a9700 1 --2- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c9019c540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009fb0 tx=0x7f7c8c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.203+0000 7f7c82ffd700 1 -- 172.21.15.47:0/384347660 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c8c0036a0 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.203+0000 7f7c82ffd700 1 -- 172.21.15.47:0/384347660 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7c8c01a430 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.203+0000 7f7c82ffd700 1 -- 172.21.15.47:0/384347660 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c8c019550 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.203+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9019cd10 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.203+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9019fa00 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.204+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9010cbc0 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.204+0000 7f7c82ffd700 1 -- 172.21.15.47:0/384347660 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7c8c019850 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.204+0000 7f7c82ffd700 1 --2- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7c7c039560 0x7f7c7c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.205+0000 7f7c82ffd700 1 -- 172.21.15.47:0/384347660 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c8c04d930 con 0x7f7c90108660 2024-04-25T20:49:34.436 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.205+0000 7f7c950a8700 1 --2- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7c7c039560 0x7f7c7c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.205+0000 7f7c950a8700 1 --2- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7c7c039560 0x7f7c7c03ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006fd0 tx=0x7f7c84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.210+0000 7f7c82ffd700 1 -- 172.21.15.47:0/384347660 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c8c00fda0 con 0x7f7c90108660 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.438+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7c90104f00 con 0x7f7c7c039560 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.442+0000 7f7c82ffd700 1 -- 172.21.15.47:0/384347660 <== mgr.14120 v2:172.21.15.47:6800/2212759557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7c90104f00 con 0x7f7c7c039560 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7c7c039560 msgr2=0x7f7c7c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 --2- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7c7c039560 0x7f7c7c03ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006fd0 tx=0x7f7c84006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 msgr2=0x7f7c9019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 --2- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c9019c540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009fb0 tx=0x7f7c8c011940 comp rx=0 tx=0).stop 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 shutdown_connections 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 --2- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f7c7c039560 0x7f7c7c03ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 --2- 172.21.15.47:0/384347660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c90108660 0x7f7c9019c540 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 >> 172.21.15.47:0/384347660 conn(0x7f7c90103c40 msgr2=0x7f7c901047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 shutdown_connections 2024-04-25T20:49:34.437 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:33.444+0000 7f7c97b0d700 1 -- 172.21.15.47:0/384347660 wait complete. 2024-04-25T20:49:34.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:34 smithi047 ceph-mon[29195]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:34.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:34 smithi047 ceph-mon[29195]: Saving service mgr spec with placement count:2 2024-04-25T20:49:34.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:34 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:34.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:34 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:35.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.855+0000 7f830bd7b700 1 Processor -- start 2024-04-25T20:49:35.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.856+0000 7f830bd7b700 1 -- start start 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.856+0000 7f830bd7b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830410b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.856+0000 7f830bd7b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8304063ce0 con 0x7f8304109130 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.856+0000 7f8309b17700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830410b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.856+0000 7f8309b17700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830410b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48842/0 (socket says 172.21.15.47:48842) 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.856+0000 7f8309b17700 1 -- 172.21.15.47:0/2970229777 learned_addr learned my addr 172.21.15.47:0/2970229777 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.856+0000 7f8309b17700 1 -- 172.21.15.47:0/2970229777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830410ba60 con 0x7f8304109130 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.856+0000 7f8309b17700 1 --2- 172.21.15.47:0/2970229777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830410b520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83000098a0 tx=0x7f8300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c58eb1d120628e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.857+0000 7f8308b15700 1 -- 172.21.15.47:0/2970229777 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8300004160 con 0x7f8304109130 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.857+0000 7f8308b15700 1 -- 172.21.15.47:0/2970229777 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8300004300 con 0x7f8304109130 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.857+0000 7f8308b15700 1 -- 172.21.15.47:0/2970229777 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8300011a20 con 0x7f8304109130 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.857+0000 7f830bd7b700 1 -- 172.21.15.47:0/2970229777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 msgr2=0x7f830410b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.857+0000 7f830bd7b700 1 --2- 172.21.15.47:0/2970229777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830410b520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83000098a0 tx=0x7f8300009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.857+0000 7f830bd7b700 1 -- 172.21.15.47:0/2970229777 shutdown_connections 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.857+0000 7f830bd7b700 1 --2- 172.21.15.47:0/2970229777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830410b520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.857+0000 7f830bd7b700 1 -- 172.21.15.47:0/2970229777 >> 172.21.15.47:0/2970229777 conn(0x7f8304102b80 msgr2=0x7f8304104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.858+0000 7f830bd7b700 1 -- 172.21.15.47:0/2970229777 shutdown_connections 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.858+0000 7f830bd7b700 1 -- 172.21.15.47:0/2970229777 wait complete. 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.858+0000 7f830bd7b700 1 Processor -- start 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.858+0000 7f830bd7b700 1 -- start start 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.858+0000 7f830bd7b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.858+0000 7f830bd7b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f830410ca00 con 0x7f8304109130 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f8309b17700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f8309b17700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48848/0 (socket says 172.21.15.47:48848) 2024-04-25T20:49:35.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f8309b17700 1 -- 172.21.15.47:0/3480953754 learned_addr learned my addr 172.21.15.47:0/3480953754 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f8309b17700 1 -- 172.21.15.47:0/3480953754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830419ca30 con 0x7f8304109130 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f8309b17700 1 --2- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830419c4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8300004660 tx=0x7f8300011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f82faffd700 1 -- 172.21.15.47:0/3480953754 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8300003aa0 con 0x7f8304109130 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f82faffd700 1 -- 172.21.15.47:0/3480953754 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830001b8f0 con 0x7f8304109130 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830419ccc0 con 0x7f8304109130 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f82faffd700 1 -- 172.21.15.47:0/3480953754 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8300023900 con 0x7f8304109130 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.859+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830419f9b0 con 0x7f8304109130 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.860+0000 7f82faffd700 1 -- 172.21.15.47:0/3480953754 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8300023b70 con 0x7f8304109130 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.860+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83040640a0 con 0x7f8304109130 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.860+0000 7f82faffd700 1 --2- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f82f0039600 0x7f82f003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.860+0000 7f82faffd700 1 -- 172.21.15.47:0/3480953754 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f830004edb0 con 0x7f8304109130 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.861+0000 7f8309316700 1 --2- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f82f0039600 0x7f82f003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.861+0000 7f8309316700 1 --2- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f82f0039600 0x7f82f003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82f4006fd0 tx=0x7f82f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:34.866+0000 7f82faffd700 1 -- 172.21.15.47:0/3480953754 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f830001f030 con 0x7f8304109130 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.075+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f830419ffa0 con 0x7f8304109130 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.078+0000 7f82faffd700 1 -- 172.21.15.47:0/3480953754 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f830001a980 con 0x7f8304109130 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.080+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f82f0039600 msgr2=0x7f82f003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.080+0000 7f830bd7b700 1 --2- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f82f0039600 0x7f82f003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82f4006fd0 tx=0x7f82f4006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.080+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 msgr2=0x7f830419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.080+0000 7f830bd7b700 1 --2- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830419c4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8300004660 tx=0x7f8300011aa0 comp rx=0 tx=0).stop 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.080+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 shutdown_connections 2024-04-25T20:49:35.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.080+0000 7f830bd7b700 1 --2- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f82f0039600 0x7f82f003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:35.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.080+0000 7f830bd7b700 1 --2- 172.21.15.47:0/3480953754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8304109130 0x7f830419c4f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:35.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.080+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 >> 172.21.15.47:0/3480953754 conn(0x7f8304102b80 msgr2=0x7f83041035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:35.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.081+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 shutdown_connections 2024-04-25T20:49:35.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:35.081+0000 7f830bd7b700 1 -- 172.21.15.47:0/3480953754 wait complete. 2024-04-25T20:49:36.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:35 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:36.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:35 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3480953754' entity='client.admin' 2024-04-25T20:49:37.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.082+0000 7ff29558f700 1 Processor -- start 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.082+0000 7ff29558f700 1 -- start start 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.082+0000 7ff29558f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff290108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.082+0000 7ff29558f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff290109060 con 0x7ff290108670 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.082+0000 7ff28effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff290108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.082+0000 7ff28effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff290108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48852/0 (socket says 172.21.15.47:48852) 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.082+0000 7ff28effd700 1 -- 172.21.15.47:0/1943154585 learned_addr learned my addr 172.21.15.47:0/1943154585 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.082+0000 7ff28effd700 1 -- 172.21.15.47:0/1943154585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff290109820 con 0x7ff290108670 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff28effd700 1 --2- 172.21.15.47:0/1943154585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff290108a90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2800098a0 tx=0x7ff280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d175dbefdba87c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff28dffb700 1 -- 172.21.15.47:0/1943154585 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff280004160 con 0x7ff290108670 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff28dffb700 1 -- 172.21.15.47:0/1943154585 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280004300 con 0x7ff290108670 2024-04-25T20:49:37.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff28dffb700 1 -- 172.21.15.47:0/1943154585 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff280011a20 con 0x7ff290108670 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff29558f700 1 -- 172.21.15.47:0/1943154585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 msgr2=0x7ff290108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff29558f700 1 --2- 172.21.15.47:0/1943154585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff290108a90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2800098a0 tx=0x7ff280009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff29558f700 1 -- 172.21.15.47:0/1943154585 shutdown_connections 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff29558f700 1 --2- 172.21.15.47:0/1943154585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff290108a90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff29558f700 1 -- 172.21.15.47:0/1943154585 >> 172.21.15.47:0/1943154585 conn(0x7ff290103c30 msgr2=0x7ff290106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff29558f700 1 -- 172.21.15.47:0/1943154585 shutdown_connections 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.083+0000 7ff29558f700 1 -- 172.21.15.47:0/1943154585 wait complete. 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.084+0000 7ff29558f700 1 Processor -- start 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.084+0000 7ff29558f700 1 -- start start 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.084+0000 7ff29558f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff29007eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.084+0000 7ff29558f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff29010cbd0 con 0x7ff290108670 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff28effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff29007eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:37.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff28effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff29007eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48864/0 (socket says 172.21.15.47:48864) 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff28effd700 1 -- 172.21.15.47:0/968111179 learned_addr learned my addr 172.21.15.47:0/968111179 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff28effd700 1 -- 172.21.15.47:0/968111179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29007f0b0 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff28effd700 1 --2- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff29007eb70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff2800119f0 tx=0x7ff280011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff277fff700 1 -- 172.21.15.47:0/968111179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2800039e0 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29007d2a0 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff277fff700 1 -- 172.21.15.47:0/968111179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28001b8f0 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff277fff700 1 -- 172.21.15.47:0/968111179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2800237a0 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.085+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29007d7e0 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.086+0000 7ff277fff700 1 -- 172.21.15.47:0/968111179 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff28001b440 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.086+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29010cd50 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.086+0000 7ff277fff700 1 --2- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7ff278039560 0x7ff27803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.086+0000 7ff277fff700 1 -- 172.21.15.47:0/968111179 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff28004de80 con 0x7ff290108670 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.086+0000 7ff28e7fc700 1 --2- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7ff278039560 0x7ff27803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:37.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.087+0000 7ff28e7fc700 1 --2- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7ff278039560 0x7ff27803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff284006fd0 tx=0x7ff284006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.091+0000 7ff277fff700 1 -- 172.21.15.47:0/968111179 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff28001f030 con 0x7ff290108670 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.310+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff2900640a0 con 0x7ff290108670 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.312+0000 7ff277fff700 1 -- 172.21.15.47:0/968111179 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff28002cdb0 con 0x7ff290108670 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7ff278039560 msgr2=0x7ff27803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 --2- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7ff278039560 0x7ff27803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff284006fd0 tx=0x7ff284006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 msgr2=0x7ff29007eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 --2- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff29007eb70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff2800119f0 tx=0x7ff280011ad0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 shutdown_connections 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 --2- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7ff278039560 0x7ff27803ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 --2- 172.21.15.47:0/968111179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff290108670 0x7ff29007eb70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 >> 172.21.15.47:0/968111179 conn(0x7ff290103c30 msgr2=0x7ff290104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 shutdown_connections 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:36.315+0000 7ff29558f700 1 -- 172.21.15.47:0/968111179 wait complete. 2024-04-25T20:49:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling the dashboard module... 2024-04-25T20:49:37.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:37 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/968111179' entity='client.admin' 2024-04-25T20:49:38.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:38 smithi047 ceph-mon[29195]: from='mgr.14120 172.21.15.47:0/450164187' entity='mgr.x' 2024-04-25T20:49:38.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:38 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1627697410' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T20:49:38.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:38 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: ignoring --setuser ceph since I am not root 2024-04-25T20:49:38.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:38 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: ignoring --setgroup ceph since I am not root 2024-04-25T20:49:38.862 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:38 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:38.791+0000 7f9889101200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:49:39.357 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:38 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:38.987+0000 7f9889101200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:49:39.357 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:39 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:39.094+0000 7f9889101200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:49:39.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.852+0000 7f9cb8b2f700 1 Processor -- start 2024-04-25T20:49:39.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.852+0000 7f9cb8b2f700 1 -- start start 2024-04-25T20:49:39.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.852+0000 7f9cb8b2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb4108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:39.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.852+0000 7f9cb8b2f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4109000 con 0x7f9cb4108610 2024-04-25T20:49:39.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.853+0000 7f9cb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb4108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.853+0000 7f9cb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb4108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48876/0 (socket says 172.21.15.47:48876) 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.853+0000 7f9cb259c700 1 -- 172.21.15.47:0/3495528646 learned_addr learned my addr 172.21.15.47:0/3495528646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.853+0000 7f9cb259c700 1 -- 172.21.15.47:0/3495528646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb4109810 con 0x7f9cb4108610 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.853+0000 7f9cb259c700 1 --2- 172.21.15.47:0/3495528646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb4108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40098a0 tx=0x7f9ca4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b1f70c8cdd25fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.853+0000 7f9cb159a700 1 -- 172.21.15.47:0/3495528646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4004160 con 0x7f9cb4108610 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.853+0000 7f9cb159a700 1 -- 172.21.15.47:0/3495528646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ca4004300 con 0x7f9cb4108610 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.853+0000 7f9cb159a700 1 -- 172.21.15.47:0/3495528646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4011a20 con 0x7f9cb4108610 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.854+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/3495528646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 msgr2=0x7f9cb4108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.854+0000 7f9cb8b2f700 1 --2- 172.21.15.47:0/3495528646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb4108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40098a0 tx=0x7f9ca4009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.855+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/3495528646 shutdown_connections 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.855+0000 7f9cb8b2f700 1 --2- 172.21.15.47:0/3495528646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb4108a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.855+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/3495528646 >> 172.21.15.47:0/3495528646 conn(0x7f9cb4103c70 msgr2=0x7f9cb4106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:39.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.855+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/3495528646 shutdown_connections 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.855+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/3495528646 wait complete. 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.855+0000 7f9cb8b2f700 1 Processor -- start 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.856+0000 7f9cb8b2f700 1 -- start start 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.856+0000 7f9cb8b2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.856+0000 7f9cb8b2f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb410cbc0 con 0x7f9cb4108610 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.856+0000 7f9cb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.856+0000 7f9cb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48880/0 (socket says 172.21.15.47:48880) 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.856+0000 7f9cb259c700 1 -- 172.21.15.47:0/1627697410 learned_addr learned my addr 172.21.15.47:0/1627697410 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.856+0000 7f9cb259c700 1 -- 172.21.15.47:0/1627697410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb419c9f0 con 0x7f9cb4108610 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.856+0000 7f9cb259c700 1 --2- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb419c4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4000c00 tx=0x7f9ca4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.857+0000 7f9ca37fe700 1 -- 172.21.15.47:0/1627697410 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4003aa0 con 0x7f9cb4108610 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.857+0000 7f9ca37fe700 1 -- 172.21.15.47:0/1627697410 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ca401b8f0 con 0x7f9cb4108610 2024-04-25T20:49:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.857+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb419cc80 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.857+0000 7f9ca37fe700 1 -- 172.21.15.47:0/1627697410 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4023890 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.857+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb419f970 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.858+0000 7f9ca37fe700 1 -- 172.21.15.47:0/1627697410 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9ca402c3f0 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.858+0000 7f9ca37fe700 1 --2- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f9c9c0395b0 0x7f9c9c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.858+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb410cd40 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.858+0000 7f9ca37fe700 1 -- 172.21.15.47:0/1627697410 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9ca404dfb0 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.858+0000 7f9cb1d9b700 1 --2- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f9c9c0395b0 0x7f9c9c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.859+0000 7f9cb1d9b700 1 --2- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f9c9c0395b0 0x7f9c9c03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006fd0 tx=0x7f9ca8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:37.868+0000 7f9ca37fe700 1 -- 172.21.15.47:0/1627697410 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ca401f030 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.140+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f9cb40640a0 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.586+0000 7f9ca37fe700 1 -- 172.21.15.47:0/1627697410 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f9ca401bda0 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.588+0000 7f9ca37fe700 1 -- 172.21.15.47:0/1627697410 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f9ca400bcb0 con 0x7f9cb4108610 2024-04-25T20:49:39.368 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f9c9c0395b0 msgr2=0x7f9c9c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 --2- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f9c9c0395b0 0x7f9c9c03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006fd0 tx=0x7f9ca8006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 msgr2=0x7f9cb419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 --2- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb419c4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4000c00 tx=0x7f9ca4011aa0 comp rx=0 tx=0).stop 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 shutdown_connections 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 --2- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f9c9c0395b0 0x7f9c9c03ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 --2- 172.21.15.47:0/1627697410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb4108610 0x7f9cb419c4b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 >> 172.21.15.47:0/1627697410 conn(0x7f9cb4103c70 msgr2=0x7f9cb4104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 shutdown_connections 2024-04-25T20:49:39.369 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:38.589+0000 7f9cb8b2f700 1 -- 172.21.15.47:0/1627697410 wait complete. 2024-04-25T20:49:39.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:39 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1627697410' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T20:49:39.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:39 smithi047 ceph-mon[29195]: mgrmap e9: x(active, since 19s) 2024-04-25T20:49:39.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:39 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:39.419+0000 7f9889101200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:49:39.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:39 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:39.508+0000 7f9889101200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:49:40.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:40 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3012590410' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T20:49:40.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:49:40.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T20:49:40.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:49:40.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T20:49:40.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.840+0000 7fabd8b98700 1 Processor -- start 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.840+0000 7fabd8b98700 1 -- start start 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.841+0000 7fabd8b98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.841+0000 7fabd8b98700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4107f30 con 0x7fabd4107540 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.841+0000 7fabd259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.841+0000 7fabd259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48912/0 (socket says 172.21.15.47:48912) 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.841+0000 7fabd259c700 1 -- 172.21.15.47:0/2411475069 learned_addr learned my addr 172.21.15.47:0/2411475069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.841+0000 7fabd259c700 1 -- 172.21.15.47:0/2411475069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd4108740 con 0x7fabd4107540 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.842+0000 7fabd259c700 1 --2- 172.21.15.47:0/2411475069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4107960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098a0 tx=0x7fabc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71030a3e39206c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.842+0000 7fabd159a700 1 -- 172.21.15.47:0/2411475069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc4004160 con 0x7fabd4107540 2024-04-25T20:49:40.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.842+0000 7fabd159a700 1 -- 172.21.15.47:0/2411475069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc4004300 con 0x7fabd4107540 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.842+0000 7fabd159a700 1 -- 172.21.15.47:0/2411475069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc4011a20 con 0x7fabd4107540 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.843+0000 7fabd8b98700 1 -- 172.21.15.47:0/2411475069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 msgr2=0x7fabd4107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.843+0000 7fabd8b98700 1 --2- 172.21.15.47:0/2411475069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4107960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098a0 tx=0x7fabc4009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.843+0000 7fabd8b98700 1 -- 172.21.15.47:0/2411475069 shutdown_connections 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.843+0000 7fabd8b98700 1 --2- 172.21.15.47:0/2411475069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4107960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.843+0000 7fabd8b98700 1 -- 172.21.15.47:0/2411475069 >> 172.21.15.47:0/2411475069 conn(0x7fabd4102b80 msgr2=0x7fabd4104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.843+0000 7fabd8b98700 1 -- 172.21.15.47:0/2411475069 shutdown_connections 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.843+0000 7fabd8b98700 1 -- 172.21.15.47:0/2411475069 wait complete. 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd8b98700 1 Processor -- start 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd8b98700 1 -- start start 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd8b98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4192620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd8b98700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fabd410baf0 con 0x7fabd4107540 2024-04-25T20:49:40.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4192620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4192620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48924/0 (socket says 172.21.15.47:48924) 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd259c700 1 -- 172.21.15.47:0/3012590410 learned_addr learned my addr 172.21.15.47:0/3012590410 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd259c700 1 -- 172.21.15.47:0/3012590410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd4192b60 con 0x7fabd4107540 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.844+0000 7fabd259c700 1 --2- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4192620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabc40119f0 tx=0x7fabc4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.845+0000 7fabc37fe700 1 -- 172.21.15.47:0/3012590410 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc40039e0 con 0x7fabd4107540 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.845+0000 7fabc37fe700 1 -- 172.21.15.47:0/3012590410 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc401b8f0 con 0x7fabd4107540 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.845+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd4190d50 con 0x7fabd4107540 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.845+0000 7fabc37fe700 1 -- 172.21.15.47:0/3012590410 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc40237a0 con 0x7fabd4107540 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.845+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd4191290 con 0x7fabd4107540 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.845+0000 7fabc37fe700 1 -- 172.21.15.47:0/3012590410 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fabc4023a70 con 0x7fabd4107540 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.846+0000 7fabc37fe700 1 --2- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fabbc039260 0x7fabbc03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.846+0000 7fabc37fe700 1 -- 172.21.15.47:0/3012590410 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fabc401baa0 con 0x7fabd4107540 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.846+0000 7fabd1d9b700 1 -- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fabbc039260 msgr2=0x7fabbc03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2212759557 2024-04-25T20:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.846+0000 7fabd1d9b700 1 --2- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fabbc039260 0x7fabbc03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.846+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd410bc70 con 0x7fabd4107540 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:39.851+0000 7fabc37fe700 1 -- 172.21.15.47:0/3012590410 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabc401f030 con 0x7fabd4107540 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.047+0000 7fabd1d9b700 1 -- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fabbc039260 msgr2=0x7fabbc03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2212759557 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.047+0000 7fabd1d9b700 1 --2- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fabbc039260 0x7fabbc03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.164+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fabd4195f30 con 0x7fabd4107540 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.164+0000 7fabc37fe700 1 -- 172.21.15.47:0/3012590410 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fabc400be60 con 0x7fabd4107540 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fabbc039260 msgr2=0x7fabbc03b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 --2- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fabbc039260 0x7fabbc03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 msgr2=0x7fabd4192620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 --2- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4192620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabc40119f0 tx=0x7fabc4011ad0 comp rx=0 tx=0).stop 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 shutdown_connections 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 --2- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7fabbc039260 0x7fabbc03b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 --2- 172.21.15.47:0/3012590410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabd4107540 0x7fabd4192620 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:40.765 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 >> 172.21.15.47:0/3012590410 conn(0x7fabd4102b80 msgr2=0x7fabd4103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:40.765 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.166+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 shutdown_connections 2024-04-25T20:49:40.765 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:40.167+0000 7fabd8b98700 1 -- 172.21.15.47:0/3012590410 wait complete. 2024-04-25T20:49:40.765 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-04-25T20:49:40.765 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 9... 2024-04-25T20:49:41.004 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:40 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:40.801+0000 7f9889101200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:49:41.004 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:40 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:40.980+0000 7f9889101200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:49:41.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:41 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:41.161+0000 7f9889101200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:49:41.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:41 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:41.466+0000 7f9889101200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:49:41.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:41 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:41.564+0000 7f9889101200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:49:41.776 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:41 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:41.774+0000 7f9889101200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:49:42.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:41 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:41.976+0000 7f9889101200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:49:42.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:42 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:42.602+0000 7f9889101200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:49:43.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:43 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:43.046+0000 7f9889101200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:49:43.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:43 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:43.147+0000 7f9889101200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:49:43.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:43 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:43.360+0000 7f9889101200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:49:44.213 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:43 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:43.928+0000 7f9889101200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:49:44.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:44.025+0000 7f9889101200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:49:44.523 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:44.283+0000 7f9889101200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:49:44.523 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:44.422+0000 7f9889101200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:49:44.523 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:44.521+0000 7f9889101200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:49:44.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:44.642+0000 7f9889101200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:49:45.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:44.953+0000 7f9889101200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:49:45.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:45.583+0000 7f9889101200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:49:45.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:49:45.808+0000 7f9889101200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:49:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: Active manager daemon x restarted 2024-04-25T20:49:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: Activating manager daemon x 2024-04-25T20:49:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T20:49:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: mgrmap e10: x(active, starting, since 0.00325321s) 2024-04-25T20:49:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:49:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:49:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:49:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:49:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T20:49:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:47.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:49:47.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:49:47.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:46 smithi047 ceph-mon[29195]: [25/Apr/2024:20:49:46] ENGINE Bus STARTING 2024-04-25T20:49:47.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:46 smithi047 ceph-mon[29195]: [25/Apr/2024:20:49:46] ENGINE Serving on http://172.21.15.47:8765 2024-04-25T20:49:47.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:46 smithi047 ceph-mon[29195]: [25/Apr/2024:20:49:46] ENGINE Serving on https://172.21.15.47:7150 2024-04-25T20:49:47.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:46 smithi047 ceph-mon[29195]: [25/Apr/2024:20:49:46] ENGINE Bus STARTED 2024-04-25T20:49:47.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:46 smithi047 ceph-mon[29195]: mgrmap e11: x(active, since 1.00578s) 2024-04-25T20:49:47.799 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.203+0000 7f6a2f141700 1 Processor -- start 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.203+0000 7f6a2f141700 1 -- start start 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.203+0000 7f6a2f141700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2810b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.203+0000 7f6a2f141700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2807ce70 con 0x7f6a281090f0 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.203+0000 7f6a2cedd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2810b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.203+0000 7f6a2cedd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2810b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48936/0 (socket says 172.21.15.47:48936) 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.203+0000 7f6a2cedd700 1 -- 172.21.15.47:0/2898794081 learned_addr learned my addr 172.21.15.47:0/2898794081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.203+0000 7f6a2cedd700 1 -- 172.21.15.47:0/2898794081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2810ba20 con 0x7f6a281090f0 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.204+0000 7f6a2cedd700 1 --2- 172.21.15.47:0/2898794081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2810b4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0098a0 tx=0x7f6a1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354c53bff6ec2f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:47.800 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.204+0000 7f6a277fe700 1 -- 172.21.15.47:0/2898794081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a1c004160 con 0x7f6a281090f0 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.204+0000 7f6a277fe700 1 -- 172.21.15.47:0/2898794081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a1c004300 con 0x7f6a281090f0 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.204+0000 7f6a277fe700 1 -- 172.21.15.47:0/2898794081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a1c011a20 con 0x7f6a281090f0 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.205+0000 7f6a2f141700 1 -- 172.21.15.47:0/2898794081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 msgr2=0x7f6a2810b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.205+0000 7f6a2f141700 1 --2- 172.21.15.47:0/2898794081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2810b4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0098a0 tx=0x7f6a1c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.205+0000 7f6a2f141700 1 -- 172.21.15.47:0/2898794081 shutdown_connections 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.205+0000 7f6a2f141700 1 --2- 172.21.15.47:0/2898794081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2810b4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.205+0000 7f6a2f141700 1 -- 172.21.15.47:0/2898794081 >> 172.21.15.47:0/2898794081 conn(0x7f6a28102b20 msgr2=0x7f6a28104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.205+0000 7f6a2f141700 1 -- 172.21.15.47:0/2898794081 shutdown_connections 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.205+0000 7f6a2f141700 1 -- 172.21.15.47:0/2898794081 wait complete. 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2f141700 1 Processor -- start 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2f141700 1 -- start start 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2f141700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:47.801 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2f141700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810ca20 con 0x7f6a281090f0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2cedd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2cedd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48950/0 (socket says 172.21.15.47:48950) 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2cedd700 1 -- 172.21.15.47:0/2500775787 learned_addr learned my addr 172.21.15.47:0/2500775787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2cedd700 1 -- 172.21.15.47:0/2500775787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2819ca10 con 0x7f6a281090f0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.206+0000 7f6a2cedd700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2819c4d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0116e0 tx=0x7f6a1c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.207+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a1c003b20 con 0x7f6a281090f0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.207+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a1c01b8f0 con 0x7f6a281090f0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.207+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a1c0238c0 con 0x7f6a281090f0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.207+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2819cca0 con 0x7f6a281090f0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.207+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2819f990 con 0x7f6a281090f0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.208+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6a1c01baa0 con 0x7f6a281090f0 2024-04-25T20:49:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.208+0000 7f6a25ffb700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 0x7f6a1803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.208+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 --> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a1803c180 con 0x7f6a180395b0 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.208+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a1c04e0a0 con 0x7f6a281090f0 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.208+0000 7f6a27fff700 1 -- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 msgr2=0x7f6a1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2212759557 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.208+0000 7f6a27fff700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 0x7f6a1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.409+0000 7f6a27fff700 1 -- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 msgr2=0x7f6a1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2212759557 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.409+0000 7f6a27fff700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 0x7f6a1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.810+0000 7f6a27fff700 1 -- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 msgr2=0x7f6a1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2212759557 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:41.810+0000 7f6a27fff700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 0x7f6a1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:42.611+0000 7f6a27fff700 1 -- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 msgr2=0x7f6a1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2212759557 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:42.611+0000 7f6a27fff700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 0x7f6a1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:44.212+0000 7f6a27fff700 1 -- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 msgr2=0x7f6a1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2212759557 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:44.212+0000 7f6a27fff700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 0x7f6a1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T20:49:47.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:45.812+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f6a1c023b70 con 0x7f6a281090f0 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:45.812+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 msgr2=0x7f6a1803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:45.812+0000 7f6a25ffb700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 0x7f6a1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.814+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f6a1c0500d0 con 0x7f6a281090f0 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.814+0000 7f6a25ffb700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a1803cfe0 0x7f6a1803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.814+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a1c017ec0 con 0x7f6a1803cfe0 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.814+0000 7f6a27fff700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a1803cfe0 0x7f6a1803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.815+0000 7f6a27fff700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a1803cfe0 0x7f6a1803f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003a10 tx=0x7f6a140092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.816+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f6a1c017ec0 con 0x7f6a1803cfe0 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.822+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6a280640a0 con 0x7f6a1803cfe0 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.822+0000 7f6a25ffb700 1 -- 172.21.15.47:0/2500775787 <== mgr.14152 v2:172.21.15.47:6800/253800519 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6a280640a0 con 0x7f6a1803cfe0 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a1803cfe0 msgr2=0x7f6a1803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a1803cfe0 0x7f6a1803f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003a10 tx=0x7f6a140092b0 comp rx=0 tx=0).stop 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 msgr2=0x7f6a2819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:47.804 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2819c4d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0116e0 tx=0x7f6a1c011aa0 comp rx=0 tx=0).stop 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 shutdown_connections 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a1803cfe0 0x7f6a1803f3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a281090f0 0x7f6a2819c4d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 --2- 172.21.15.47:0/2500775787 >> [v2:172.21.15.47:6800/2212759557,v1:172.21.15.47:6801/2212759557] conn(0x7f6a180395b0 0x7f6a1803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 >> 172.21.15.47:0/2500775787 conn(0x7f6a28102b20 msgr2=0x7f6a281035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 shutdown_connections 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:46.823+0000 7f6a2f141700 1 -- 172.21.15.47:0/2500775787 wait complete. 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 9 is available 2024-04-25T20:49:47.805 INFO:teuthology.orchestra.run.smithi047.stdout:Generating a dashboard self-signed certificate... 2024-04-25T20:49:48.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:48 smithi047 ceph-mon[29195]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T20:49:48.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:48 smithi047 ceph-mon[29195]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T20:49:48.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:48 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.232+0000 7f0480368700 1 Processor -- start 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.232+0000 7f0480368700 1 -- start start 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.233+0000 7f0480368700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f0478107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.233+0000 7f0480368700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0478107f30 con 0x7f0478107540 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.233+0000 7f047e104700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f0478107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.233+0000 7f047e104700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f0478107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41406/0 (socket says 172.21.15.47:41406) 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.233+0000 7f047e104700 1 -- 172.21.15.47:0/949786142 learned_addr learned my addr 172.21.15.47:0/949786142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.233+0000 7f047e104700 1 -- 172.21.15.47:0/949786142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0478108740 con 0x7f0478107540 2024-04-25T20:49:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.234+0000 7f047e104700 1 --2- 172.21.15.47:0/949786142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f0478107960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f047400bb40 tx=0x7f047400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d173ca654c0dd92d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.234+0000 7f047d102700 1 -- 172.21.15.47:0/949786142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f047400dbe0 con 0x7f0478107540 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.234+0000 7f047d102700 1 -- 172.21.15.47:0/949786142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04740044e0 con 0x7f0478107540 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.234+0000 7f047d102700 1 -- 172.21.15.47:0/949786142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474010470 con 0x7f0478107540 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.234+0000 7f0480368700 1 -- 172.21.15.47:0/949786142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 msgr2=0x7f0478107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.234+0000 7f0480368700 1 --2- 172.21.15.47:0/949786142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f0478107960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f047400bb40 tx=0x7f047400be50 comp rx=0 tx=0).stop 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 -- 172.21.15.47:0/949786142 shutdown_connections 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 --2- 172.21.15.47:0/949786142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f0478107960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 -- 172.21.15.47:0/949786142 >> 172.21.15.47:0/949786142 conn(0x7f0478102b80 msgr2=0x7f0478104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 -- 172.21.15.47:0/949786142 shutdown_connections 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 -- 172.21.15.47:0/949786142 wait complete. 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 Processor -- start 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 -- start start 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f047819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:49.224 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.235+0000 7f0480368700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0478062e80 con 0x7f0478107540 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f047e104700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f047819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f047e104700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f047819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41412/0 (socket says 172.21.15.47:41412) 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f047e104700 1 -- 172.21.15.47:0/2600167046 learned_addr learned my addr 172.21.15.47:0/2600167046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f047e104700 1 -- 172.21.15.47:0/2600167046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047819a820 con 0x7f0478107540 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f047e104700 1 --2- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f047819a2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f047400b340 tx=0x7f0474003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f046f7fe700 1 -- 172.21.15.47:0/2600167046 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474027070 con 0x7f0478107540 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f046f7fe700 1 -- 172.21.15.47:0/2600167046 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0474010c20 con 0x7f0478107540 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f046f7fe700 1 -- 172.21.15.47:0/2600167046 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474022b70 con 0x7f0478107540 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047819aab0 con 0x7f0478107540 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.236+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047819d7a0 con 0x7f0478107540 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.237+0000 7f046f7fe700 1 -- 172.21.15.47:0/2600167046 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f047402a070 con 0x7f0478107540 2024-04-25T20:49:49.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.237+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04780640a0 con 0x7f0478107540 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.238+0000 7f046f7fe700 1 --2- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f04640394f0 0x7f046403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.238+0000 7f046f7fe700 1 -- 172.21.15.47:0/2600167046 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f047401f070 con 0x7f0478107540 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.238+0000 7f047d903700 1 --2- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f04640394f0 0x7f046403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.238+0000 7f047d903700 1 --2- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f04640394f0 0x7f046403b9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0468006fd0 tx=0x7f0468006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.244+0000 7f046f7fe700 1 -- 172.21.15.47:0/2600167046 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0474015070 con 0x7f0478107540 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.457+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f04780754c0 con 0x7f04640394f0 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.475+0000 7f046f7fe700 1 -- 172.21.15.47:0/2600167046 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f047402cd00 con 0x7f0478107540 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.625+0000 7f046f7fe700 1 -- 172.21.15.47:0/2600167046 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f04780754c0 con 0x7f04640394f0 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f04640394f0 msgr2=0x7f046403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 --2- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f04640394f0 0x7f046403b9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0468006fd0 tx=0x7f0468006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:49.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 msgr2=0x7f047819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:49.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 --2- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f047819a2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f047400b340 tx=0x7f0474003dc0 comp rx=0 tx=0).stop 2024-04-25T20:49:49.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 shutdown_connections 2024-04-25T20:49:49.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 --2- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f04640394f0 0x7f046403b9b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:49.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 --2- 172.21.15.47:0/2600167046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0478107540 0x7f047819a2e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:49.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 >> 172.21.15.47:0/2600167046 conn(0x7f0478102b80 msgr2=0x7f0478190900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:49.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 shutdown_connections 2024-04-25T20:49:49.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:48.627+0000 7f0480368700 1 -- 172.21.15.47:0/2600167046 wait complete. 2024-04-25T20:49:49.227 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial admin user... 2024-04-25T20:49:49.853 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:49 smithi047 ceph-mon[29195]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:49.853 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:49 smithi047 ceph-mon[29195]: mgrmap e12: x(active, since 2s) 2024-04-25T20:49:49.853 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:49.853 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:49.853 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:50.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$N5pKMwVgnKVJylS9jVXwMexI3AtXEXIRzbKnvpbFDc0s/49dAZNiu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714078190, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T20:49:50.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.639+0000 7effdad93700 1 Processor -- start 2024-04-25T20:49:50.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.639+0000 7effdad93700 1 -- start start 2024-04-25T20:49:50.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.639+0000 7effdad93700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd4108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:50.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.639+0000 7effdad93700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7effd4109060 con 0x7effd4108670 2024-04-25T20:49:50.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.639+0000 7effd8b2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd4108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:50.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.639+0000 7effd8b2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd4108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41418/0 (socket says 172.21.15.47:41418) 2024-04-25T20:49:50.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.639+0000 7effd8b2f700 1 -- 172.21.15.47:0/2415689264 learned_addr learned my addr 172.21.15.47:0/2415689264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.639+0000 7effd8b2f700 1 -- 172.21.15.47:0/2415689264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd4109820 con 0x7effd4108670 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effd8b2f700 1 --2- 172.21.15.47:0/2415689264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd4108a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7effc80098a0 tx=0x7effc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf29bd4e3e166b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effd37fe700 1 -- 172.21.15.47:0/2415689264 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effc8004160 con 0x7effd4108670 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effd37fe700 1 -- 172.21.15.47:0/2415689264 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc8004300 con 0x7effd4108670 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effd37fe700 1 -- 172.21.15.47:0/2415689264 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effc8011a20 con 0x7effd4108670 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effdad93700 1 -- 172.21.15.47:0/2415689264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 msgr2=0x7effd4108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effdad93700 1 --2- 172.21.15.47:0/2415689264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd4108a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7effc80098a0 tx=0x7effc8009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effdad93700 1 -- 172.21.15.47:0/2415689264 shutdown_connections 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effdad93700 1 --2- 172.21.15.47:0/2415689264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd4108a90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effdad93700 1 -- 172.21.15.47:0/2415689264 >> 172.21.15.47:0/2415689264 conn(0x7effd4103c30 msgr2=0x7effd4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effdad93700 1 -- 172.21.15.47:0/2415689264 shutdown_connections 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.640+0000 7effdad93700 1 -- 172.21.15.47:0/2415689264 wait complete. 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.641+0000 7effdad93700 1 Processor -- start 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.641+0000 7effdad93700 1 -- start start 2024-04-25T20:49:50.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.641+0000 7effdad93700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.641+0000 7effdad93700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7effd410cbd0 con 0x7effd4108670 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.641+0000 7effd8b2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.641+0000 7effd8b2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41424/0 (socket says 172.21.15.47:41424) 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.641+0000 7effd8b2f700 1 -- 172.21.15.47:0/293329174 learned_addr learned my addr 172.21.15.47:0/293329174 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.642+0000 7effd8b2f700 1 -- 172.21.15.47:0/293329174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd419ca30 con 0x7effd4108670 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.642+0000 7effd8b2f700 1 --2- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd419c4f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7effc80119f0 tx=0x7effc8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.642+0000 7effd1ffb700 1 -- 172.21.15.47:0/293329174 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effc80039e0 con 0x7effd4108670 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.642+0000 7effd1ffb700 1 -- 172.21.15.47:0/293329174 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc801b8f0 con 0x7effd4108670 2024-04-25T20:49:50.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.642+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd419ccc0 con 0x7effd4108670 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.642+0000 7effd1ffb700 1 -- 172.21.15.47:0/293329174 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effc80237a0 con 0x7effd4108670 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.642+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd419f9b0 con 0x7effd4108670 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.643+0000 7effd1ffb700 1 -- 172.21.15.47:0/293329174 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7effc8023a50 con 0x7effd4108670 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.643+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd410cd50 con 0x7effd4108670 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.643+0000 7effd1ffb700 1 --2- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7effc4039610 0x7effc403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.643+0000 7effd1ffb700 1 -- 172.21.15.47:0/293329174 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7effc804e170 con 0x7effd4108670 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.644+0000 7effd3fff700 1 --2- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7effc4039610 0x7effc403bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.644+0000 7effd3fff700 1 --2- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7effc4039610 0x7effc403bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effc0006fd0 tx=0x7effc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.648+0000 7effd1ffb700 1 -- 172.21.15.47:0/293329174 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effc801f030 con 0x7effd4108670 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:49.888+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7effd4104ea0 con 0x7effc4039610 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.134+0000 7effd1ffb700 1 -- 172.21.15.47:0/293329174 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7effd4104ea0 con 0x7effc4039610 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7effc4039610 msgr2=0x7effc403bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 --2- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7effc4039610 0x7effc403bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effc0006fd0 tx=0x7effc0006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 msgr2=0x7effd419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 --2- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd419c4f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7effc80119f0 tx=0x7effc8011ad0 comp rx=0 tx=0).stop 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 shutdown_connections 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 --2- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7effc4039610 0x7effc403bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 --2- 172.21.15.47:0/293329174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd4108670 0x7effd419c4f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 >> 172.21.15.47:0/293329174 conn(0x7effd4103c30 msgr2=0x7effd4104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 shutdown_connections 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:50.136+0000 7effdad93700 1 -- 172.21.15.47:0/293329174 wait complete. 2024-04-25T20:49:50.920 INFO:teuthology.orchestra.run.smithi047.stdout:Fetching dashboard port number... 2024-04-25T20:49:51.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:51 smithi047 ceph-mon[29195]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:51.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:52.432 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:52 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1699251071' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.615+0000 7f6f24f41700 1 Processor -- start 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.615+0000 7f6f24f41700 1 -- start start 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.615+0000 7f6f24f41700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.615+0000 7f6f24f41700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20107ee0 con 0x7f6f201074f0 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.616+0000 7f6f1e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.616+0000 7f6f1e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41440/0 (socket says 172.21.15.47:41440) 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.616+0000 7f6f1e59c700 1 -- 172.21.15.47:0/2775181517 learned_addr learned my addr 172.21.15.47:0/2775181517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.616+0000 7f6f1e59c700 1 -- 172.21.15.47:0/2775181517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f201086f0 con 0x7f6f201074f0 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.616+0000 7f6f1e59c700 1 --2- 172.21.15.47:0/2775181517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20107910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f100098a0 tx=0x7f6f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8f40e83ba182ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:52.440 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.616+0000 7f6f1d59a700 1 -- 172.21.15.47:0/2775181517 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f10004160 con 0x7f6f201074f0 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.616+0000 7f6f1d59a700 1 -- 172.21.15.47:0/2775181517 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f10004300 con 0x7f6f201074f0 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.616+0000 7f6f1d59a700 1 -- 172.21.15.47:0/2775181517 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f10011a20 con 0x7f6f201074f0 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.617+0000 7f6f24f41700 1 -- 172.21.15.47:0/2775181517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 msgr2=0x7f6f20107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.617+0000 7f6f24f41700 1 --2- 172.21.15.47:0/2775181517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20107910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f100098a0 tx=0x7f6f10009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.617+0000 7f6f24f41700 1 -- 172.21.15.47:0/2775181517 shutdown_connections 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.617+0000 7f6f24f41700 1 --2- 172.21.15.47:0/2775181517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20107910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.617+0000 7f6f24f41700 1 -- 172.21.15.47:0/2775181517 >> 172.21.15.47:0/2775181517 conn(0x7f6f20102af0 msgr2=0x7f6f20104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.617+0000 7f6f24f41700 1 -- 172.21.15.47:0/2775181517 shutdown_connections 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.617+0000 7f6f24f41700 1 -- 172.21.15.47:0/2775181517 wait complete. 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f24f41700 1 Processor -- start 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f24f41700 1 -- start start 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f24f41700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20192580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f24f41700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010baa0 con 0x7f6f201074f0 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f1e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20192580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f1e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20192580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41444/0 (socket says 172.21.15.47:41444) 2024-04-25T20:49:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f1e59c700 1 -- 172.21.15.47:0/1699251071 learned_addr learned my addr 172.21.15.47:0/1699251071 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f1e59c700 1 -- 172.21.15.47:0/1699251071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f20192ac0 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.618+0000 7f6f1e59c700 1 --2- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20192580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f100119f0 tx=0x7f6f10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.619+0000 7f6f0f7fe700 1 -- 172.21.15.47:0/1699251071 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f100039e0 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.619+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f20190d00 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.619+0000 7f6f0f7fe700 1 -- 172.21.15.47:0/1699251071 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1001b8f0 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.619+0000 7f6f0f7fe700 1 -- 172.21.15.47:0/1699251071 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f100237a0 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.619+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f201911e0 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.620+0000 7f6f0f7fe700 1 -- 172.21.15.47:0/1699251071 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6f1002c3f0 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.620+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2010bc20 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.620+0000 7f6f0f7fe700 1 --2- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6f080395b0 0x7f6f0803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.620+0000 7f6f0f7fe700 1 -- 172.21.15.47:0/1699251071 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f1004e130 con 0x7f6f201074f0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.620+0000 7f6f1dd9b700 1 --2- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6f080395b0 0x7f6f0803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.620+0000 7f6f1dd9b700 1 --2- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6f080395b0 0x7f6f0803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6f14006fd0 tx=0x7f6f14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.625+0000 7f6f0f7fe700 1 -- 172.21.15.47:0/1699251071 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f1001f030 con 0x7f6f201074f0 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.832+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f6f20196290 con 0x7f6f201074f0 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.832+0000 7f6f0f7fe700 1 -- 172.21.15.47:0/1699251071 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6f1001bb10 con 0x7f6f201074f0 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6f080395b0 msgr2=0x7f6f0803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 --2- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6f080395b0 0x7f6f0803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6f14006fd0 tx=0x7f6f14006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 msgr2=0x7f6f20192580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 --2- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20192580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f100119f0 tx=0x7f6f10011ad0 comp rx=0 tx=0).stop 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 shutdown_connections 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 --2- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6f080395b0 0x7f6f0803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 --2- 172.21.15.47:0/1699251071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f201074f0 0x7f6f20192580 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 >> 172.21.15.47:0/1699251071 conn(0x7f6f20102af0 msgr2=0x7f6f201039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 shutdown_connections 2024-04-25T20:49:52.443 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:51.835+0000 7f6f24f41700 1 -- 172.21.15.47:0/1699251071 wait complete. 2024-04-25T20:49:52.447 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:49:52.447 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-04-25T20:49:52.455 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:49:52.455 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-25T20:49:52.455 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-04-25T20:49:52.455 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T20:49:52.457 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph Dashboard is now available at: 2024-04-25T20:49:52.457 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:52.457 INFO:teuthology.orchestra.run.smithi047.stdout: URL: https://smithi047:8443/ 2024-04-25T20:49:52.457 INFO:teuthology.orchestra.run.smithi047.stdout: User: admin 2024-04-25T20:49:52.457 INFO:teuthology.orchestra.run.smithi047.stdout: Password: 8fp281dzpj 2024-04-25T20:49:52.457 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:52.457 INFO:teuthology.orchestra.run.smithi047.stdout:Saving cluster configuration to /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config directory 2024-04-25T20:49:52.458 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling autotune for osd_memory_target 2024-04-25T20:49:53.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:53 smithi047 ceph-mon[29195]: mgrmap e13: x(active, since 6s) 2024-04-25T20:49:53.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.830+0000 7f8726e5c700 1 Processor -- start 2024-04-25T20:49:53.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.830+0000 7f8726e5c700 1 -- start start 2024-04-25T20:49:53.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.830+0000 7f8726e5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f8720108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:53.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.830+0000 7f8726e5c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8720108ff0 con 0x7f8720108600 2024-04-25T20:49:53.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.831+0000 7f8724bf8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f8720108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:53.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.831+0000 7f8724bf8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f8720108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41456/0 (socket says 172.21.15.47:41456) 2024-04-25T20:49:53.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.831+0000 7f8724bf8700 1 -- 172.21.15.47:0/3149562188 learned_addr learned my addr 172.21.15.47:0/3149562188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:53.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.831+0000 7f8724bf8700 1 -- 172.21.15.47:0/3149562188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8720109800 con 0x7f8720108600 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.831+0000 7f8724bf8700 1 --2- 172.21.15.47:0/3149562188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f8720108a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87140098a0 tx=0x7f8714009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607748acecacf28e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.831+0000 7f871f7fe700 1 -- 172.21.15.47:0/3149562188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8714004160 con 0x7f8720108600 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.831+0000 7f871f7fe700 1 -- 172.21.15.47:0/3149562188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8714004300 con 0x7f8720108600 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.831+0000 7f871f7fe700 1 -- 172.21.15.47:0/3149562188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8714011a20 con 0x7f8720108600 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 -- 172.21.15.47:0/3149562188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 msgr2=0x7f8720108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 --2- 172.21.15.47:0/3149562188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f8720108a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87140098a0 tx=0x7f8714009bb0 comp rx=0 tx=0).stop 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 -- 172.21.15.47:0/3149562188 shutdown_connections 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 --2- 172.21.15.47:0/3149562188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f8720108a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 -- 172.21.15.47:0/3149562188 >> 172.21.15.47:0/3149562188 conn(0x7f8720103be0 msgr2=0x7f8720106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 -- 172.21.15.47:0/3149562188 shutdown_connections 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 -- 172.21.15.47:0/3149562188 wait complete. 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 Processor -- start 2024-04-25T20:49:53.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.832+0000 7f8726e5c700 1 -- start start 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f8726e5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f872007eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f8726e5c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f872010cbb0 con 0x7f8720108600 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f8724bf8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f872007eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f8724bf8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f872007eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41458/0 (socket says 172.21.15.47:41458) 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f8724bf8700 1 -- 172.21.15.47:0/651550578 learned_addr learned my addr 172.21.15.47:0/651550578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f8724bf8700 1 -- 172.21.15.47:0/651550578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872007f0b0 con 0x7f8720108600 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f8724bf8700 1 --2- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f872007eb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87140119f0 tx=0x7f8714011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f871dffb700 1 -- 172.21.15.47:0/651550578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87140039e0 con 0x7f8720108600 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f871dffb700 1 -- 172.21.15.47:0/651550578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f871401b8f0 con 0x7f8720108600 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f871dffb700 1 -- 172.21.15.47:0/651550578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87140237a0 con 0x7f8720108600 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.833+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872007d2a0 con 0x7f8720108600 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.834+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872007d7e0 con 0x7f8720108600 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.834+0000 7f871dffb700 1 -- 172.21.15.47:0/651550578 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f871401b440 con 0x7f8720108600 2024-04-25T20:49:53.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.834+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872010cd30 con 0x7f8720108600 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.835+0000 7f871dffb700 1 --2- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f87100396b0 0x7f871003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.835+0000 7f871dffb700 1 -- 172.21.15.47:0/651550578 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f871404e010 con 0x7f8720108600 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.835+0000 7f871ffff700 1 --2- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f87100396b0 0x7f871003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.835+0000 7f871ffff700 1 --2- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f87100396b0 0x7f871003bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f870c006fd0 tx=0x7f870c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:52.840+0000 7f871dffb700 1 -- 172.21.15.47:0/651550578 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f871401f030 con 0x7f8720108600 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.044+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f87200640a0 con 0x7f8720108600 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.045+0000 7f871dffb700 1 -- 172.21.15.47:0/651550578 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f8714018350 con 0x7f8720108600 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f87100396b0 msgr2=0x7f871003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 --2- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f87100396b0 0x7f871003bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f870c006fd0 tx=0x7f870c006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 msgr2=0x7f872007eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 --2- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f872007eb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87140119f0 tx=0x7f8714011ad0 comp rx=0 tx=0).stop 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 shutdown_connections 2024-04-25T20:49:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 --2- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f87100396b0 0x7f871003bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:53.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 --2- 172.21.15.47:0/651550578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720108600 0x7f872007eb70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:53.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 >> 172.21.15.47:0/651550578 conn(0x7f8720103be0 msgr2=0x7f8720104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:53.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 shutdown_connections 2024-04-25T20:49:53.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:53.047+0000 7f8726e5c700 1 -- 172.21.15.47:0/651550578 wait complete. 2024-04-25T20:49:54.895 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.058+0000 7f9b2abac700 1 Processor -- start 2024-04-25T20:49:54.895 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.058+0000 7f9b2abac700 1 -- start start 2024-04-25T20:49:54.895 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.058+0000 7f9b2abac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b24108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:54.895 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.058+0000 7f9b2abac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24109060 con 0x7f9b24108670 2024-04-25T20:49:54.895 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.059+0000 7f9b28948700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b24108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.059+0000 7f9b28948700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b24108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53016/0 (socket says 172.21.15.47:53016) 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.059+0000 7f9b28948700 1 -- 172.21.15.47:0/1223489545 learned_addr learned my addr 172.21.15.47:0/1223489545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.059+0000 7f9b28948700 1 -- 172.21.15.47:0/1223489545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b24109870 con 0x7f9b24108670 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.059+0000 7f9b28948700 1 --2- 172.21.15.47:0/1223489545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b24108a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009e80 tx=0x7f9b1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae343798938ddb76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.059+0000 7f9b237fe700 1 -- 172.21.15.47:0/1223489545 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c017070 con 0x7f9b24108670 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.059+0000 7f9b237fe700 1 -- 172.21.15.47:0/1223489545 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b1c00b8c0 con 0x7f9b24108670 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.059+0000 7f9b237fe700 1 -- 172.21.15.47:0/1223489545 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c004490 con 0x7f9b24108670 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.060+0000 7f9b2abac700 1 -- 172.21.15.47:0/1223489545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 msgr2=0x7f9b24108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.060+0000 7f9b2abac700 1 --2- 172.21.15.47:0/1223489545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b24108a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009e80 tx=0x7f9b1c00b330 comp rx=0 tx=0).stop 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.060+0000 7f9b2abac700 1 -- 172.21.15.47:0/1223489545 shutdown_connections 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.060+0000 7f9b2abac700 1 --2- 172.21.15.47:0/1223489545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b24108a90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.060+0000 7f9b2abac700 1 -- 172.21.15.47:0/1223489545 >> 172.21.15.47:0/1223489545 conn(0x7f9b24103c30 msgr2=0x7f9b24106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.060+0000 7f9b2abac700 1 -- 172.21.15.47:0/1223489545 shutdown_connections 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.060+0000 7f9b2abac700 1 -- 172.21.15.47:0/1223489545 wait complete. 2024-04-25T20:49:54.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b2abac700 1 Processor -- start 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b2abac700 1 -- start start 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b2abac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b2419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b2abac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2410cc20 con 0x7f9b24108670 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b28948700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b2419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b28948700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b2419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53028/0 (socket says 172.21.15.47:53028) 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b28948700 1 -- 172.21.15.47:0/3355963651 learned_addr learned my addr 172.21.15.47:0/3355963651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b28948700 1 -- 172.21.15.47:0/3355963651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2419ec70 con 0x7f9b24108670 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b28948700 1 --2- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b2419e730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009e50 tx=0x7f9b1c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.061+0000 7f9b21ffb700 1 -- 172.21.15.47:0/3355963651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c020020 con 0x7f9b24108670 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.062+0000 7f9b21ffb700 1 -- 172.21.15.47:0/3355963651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b1c003b50 con 0x7f9b24108670 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.062+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2419ef00 con 0x7f9b24108670 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.062+0000 7f9b21ffb700 1 -- 172.21.15.47:0/3355963651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c017040 con 0x7f9b24108670 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.062+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241a1bf0 con 0x7f9b24108670 2024-04-25T20:49:54.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.063+0000 7f9b21ffb700 1 -- 172.21.15.47:0/3355963651 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9b1c023020 con 0x7f9b24108670 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.063+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b240640a0 con 0x7f9b24108670 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.063+0000 7f9b21ffb700 1 --2- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9b0c039650 0x7f9b0c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.063+0000 7f9b21ffb700 1 -- 172.21.15.47:0/3355963651 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9b1c00f050 con 0x7f9b24108670 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.063+0000 7f9b23fff700 1 --2- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9b0c039650 0x7f9b0c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.064+0000 7f9b23fff700 1 --2- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9b0c039650 0x7f9b0c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b18006fd0 tx=0x7f9b18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.068+0000 7f9b21ffb700 1 -- 172.21.15.47:0/3355963651 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b1c04fe20 con 0x7f9b24108670 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.391+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f9b241a21c0 con 0x7f9b24108670 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.393+0000 7f9b21ffb700 1 -- 172.21.15.47:0/3355963651 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f9b1c02a020 con 0x7f9b24108670 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.395+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9b0c039650 msgr2=0x7f9b0c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.395+0000 7f9b2abac700 1 --2- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9b0c039650 0x7f9b0c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b18006fd0 tx=0x7f9b18006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.395+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 msgr2=0x7f9b2419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.395+0000 7f9b2abac700 1 --2- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b2419e730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009e50 tx=0x7f9b1c00bd50 comp rx=0 tx=0).stop 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.395+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 shutdown_connections 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.395+0000 7f9b2abac700 1 --2- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9b0c039650 0x7f9b0c03bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.395+0000 7f9b2abac700 1 --2- 172.21.15.47:0/3355963651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b24108670 0x7f9b2419e730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.395+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 >> 172.21.15.47:0/3355963651 conn(0x7f9b24103c30 msgr2=0x7f9b24104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.396+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 shutdown_connections 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-25T20:49:54.396+0000 7f9b2abac700 1 -- 172.21.15.47:0/3355963651 wait complete. 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T20:49:54.899 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph telemetry on 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout:For more information see: 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:49:54.900 INFO:teuthology.orchestra.run.smithi047.stdout:Bootstrap complete. 2024-04-25T20:49:54.932 INFO:tasks.cephadm:Fetching config... 2024-04-25T20:49:54.932 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:49:54.932 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T20:49:54.948 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T20:49:54.948 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:49:54.949 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T20:49:55.005 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T20:49:55.005 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:49:55.005 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T20:49:55.069 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T20:49:55.070 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:49:55.070 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T20:49:55.125 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T20:49:55.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9Njub0SmJWpO26SeOpG/cUp4ezSVV+Yo7UTuMz/Z4Xk/EBMitfnBO+NMiuLQG+TRA3SBkhtup3f+5lNcQGhDqeX4qSPGFVRf6VVH7i3iRKa5tyqYnTONlOn2T015+q1IaccIsYiVXIyaBQttz3A/zrKoaJwkxSoptXP+iU/WHzai+GqUYmYo6acUKrWkR8oW6vVrgQ16c0UWP8g69kX1pDzHuJl9s1I8N0O08tMLatYhkVrBGfwaAmdb0DWjYnXGIp5Yfp0JKoiiZMDa00ZE/Fw+yqyTpVV5ww3tpAu2dw/pF6TDMOAmEVN+8wUTzNsOc4ZXWTYAGruwn4CUXATlmlPwRJWnkXgrkQu2jKRFgVJfYndrZK+OX9YMpEpkVlAFOJeuv4uJGeLBZzVA93uScBKsM2Gqre2nkseAUKJNjk3S31p/kNAkSFa7yTm6FiQpkhFjjLuVDPupd1m1QOJNvz0PklHKqExUv+dudXkiGNrxHmqzM0kVo9wv8Wnp4wnE= ceph-0ca8de06-0345-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:49:55.197 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9Njub0SmJWpO26SeOpG/cUp4ezSVV+Yo7UTuMz/Z4Xk/EBMitfnBO+NMiuLQG+TRA3SBkhtup3f+5lNcQGhDqeX4qSPGFVRf6VVH7i3iRKa5tyqYnTONlOn2T015+q1IaccIsYiVXIyaBQttz3A/zrKoaJwkxSoptXP+iU/WHzai+GqUYmYo6acUKrWkR8oW6vVrgQ16c0UWP8g69kX1pDzHuJl9s1I8N0O08tMLatYhkVrBGfwaAmdb0DWjYnXGIp5Yfp0JKoiiZMDa00ZE/Fw+yqyTpVV5ww3tpAu2dw/pF6TDMOAmEVN+8wUTzNsOc4ZXWTYAGruwn4CUXATlmlPwRJWnkXgrkQu2jKRFgVJfYndrZK+OX9YMpEpkVlAFOJeuv4uJGeLBZzVA93uScBKsM2Gqre2nkseAUKJNjk3S31p/kNAkSFa7yTm6FiQpkhFjjLuVDPupd1m1QOJNvz0PklHKqExUv+dudXkiGNrxHmqzM0kVo9wv8Wnp4wnE= ceph-0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:49:55.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9Njub0SmJWpO26SeOpG/cUp4ezSVV+Yo7UTuMz/Z4Xk/EBMitfnBO+NMiuLQG+TRA3SBkhtup3f+5lNcQGhDqeX4qSPGFVRf6VVH7i3iRKa5tyqYnTONlOn2T015+q1IaccIsYiVXIyaBQttz3A/zrKoaJwkxSoptXP+iU/WHzai+GqUYmYo6acUKrWkR8oW6vVrgQ16c0UWP8g69kX1pDzHuJl9s1I8N0O08tMLatYhkVrBGfwaAmdb0DWjYnXGIp5Yfp0JKoiiZMDa00ZE/Fw+yqyTpVV5ww3tpAu2dw/pF6TDMOAmEVN+8wUTzNsOc4ZXWTYAGruwn4CUXATlmlPwRJWnkXgrkQu2jKRFgVJfYndrZK+OX9YMpEpkVlAFOJeuv4uJGeLBZzVA93uScBKsM2Gqre2nkseAUKJNjk3S31p/kNAkSFa7yTm6FiQpkhFjjLuVDPupd1m1QOJNvz0PklHKqExUv+dudXkiGNrxHmqzM0kVo9wv8Wnp4wnE= ceph-0ca8de06-0345-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:49:55.248 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9Njub0SmJWpO26SeOpG/cUp4ezSVV+Yo7UTuMz/Z4Xk/EBMitfnBO+NMiuLQG+TRA3SBkhtup3f+5lNcQGhDqeX4qSPGFVRf6VVH7i3iRKa5tyqYnTONlOn2T015+q1IaccIsYiVXIyaBQttz3A/zrKoaJwkxSoptXP+iU/WHzai+GqUYmYo6acUKrWkR8oW6vVrgQ16c0UWP8g69kX1pDzHuJl9s1I8N0O08tMLatYhkVrBGfwaAmdb0DWjYnXGIp5Yfp0JKoiiZMDa00ZE/Fw+yqyTpVV5ww3tpAu2dw/pF6TDMOAmEVN+8wUTzNsOc4ZXWTYAGruwn4CUXATlmlPwRJWnkXgrkQu2jKRFgVJfYndrZK+OX9YMpEpkVlAFOJeuv4uJGeLBZzVA93uScBKsM2Gqre2nkseAUKJNjk3S31p/kNAkSFa7yTm6FiQpkhFjjLuVDPupd1m1QOJNvz0PklHKqExUv+dudXkiGNrxHmqzM0kVo9wv8Wnp4wnE= ceph-0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:49:55.259 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9Njub0SmJWpO26SeOpG/cUp4ezSVV+Yo7UTuMz/Z4Xk/EBMitfnBO+NMiuLQG+TRA3SBkhtup3f+5lNcQGhDqeX4qSPGFVRf6VVH7i3iRKa5tyqYnTONlOn2T015+q1IaccIsYiVXIyaBQttz3A/zrKoaJwkxSoptXP+iU/WHzai+GqUYmYo6acUKrWkR8oW6vVrgQ16c0UWP8g69kX1pDzHuJl9s1I8N0O08tMLatYhkVrBGfwaAmdb0DWjYnXGIp5Yfp0JKoiiZMDa00ZE/Fw+yqyTpVV5ww3tpAu2dw/pF6TDMOAmEVN+8wUTzNsOc4ZXWTYAGruwn4CUXATlmlPwRJWnkXgrkQu2jKRFgVJfYndrZK+OX9YMpEpkVlAFOJeuv4uJGeLBZzVA93uScBKsM2Gqre2nkseAUKJNjk3S31p/kNAkSFa7yTm6FiQpkhFjjLuVDPupd1m1QOJNvz0PklHKqExUv+dudXkiGNrxHmqzM0kVo9wv8Wnp4wnE= ceph-0ca8de06-0345-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:49:55.299 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9Njub0SmJWpO26SeOpG/cUp4ezSVV+Yo7UTuMz/Z4Xk/EBMitfnBO+NMiuLQG+TRA3SBkhtup3f+5lNcQGhDqeX4qSPGFVRf6VVH7i3iRKa5tyqYnTONlOn2T015+q1IaccIsYiVXIyaBQttz3A/zrKoaJwkxSoptXP+iU/WHzai+GqUYmYo6acUKrWkR8oW6vVrgQ16c0UWP8g69kX1pDzHuJl9s1I8N0O08tMLatYhkVrBGfwaAmdb0DWjYnXGIp5Yfp0JKoiiZMDa00ZE/Fw+yqyTpVV5ww3tpAu2dw/pF6TDMOAmEVN+8wUTzNsOc4ZXWTYAGruwn4CUXATlmlPwRJWnkXgrkQu2jKRFgVJfYndrZK+OX9YMpEpkVlAFOJeuv4uJGeLBZzVA93uScBKsM2Gqre2nkseAUKJNjk3S31p/kNAkSFa7yTm6FiQpkhFjjLuVDPupd1m1QOJNvz0PklHKqExUv+dudXkiGNrxHmqzM0kVo9wv8Wnp4wnE= ceph-0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:49:55.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T20:49:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:49:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:55 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3355963651' entity='client.admin' 2024-04-25T20:49:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.708+0000 7f4560898700 1 -- 172.21.15.47:0/829043494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 msgr2=0x7f455c1017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.708+0000 7f4560898700 1 --2- 172.21.15.47:0/829043494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 0x7f455c1017e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f454c009910 tx=0x7f454c009c20 comp rx=0 tx=0).stop 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 -- 172.21.15.47:0/829043494 shutdown_connections 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 --2- 172.21.15.47:0/829043494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 0x7f455c1017e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 -- 172.21.15.47:0/829043494 >> 172.21.15.47:0/829043494 conn(0x7f455c0fc980 msgr2=0x7f455c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 -- 172.21.15.47:0/829043494 shutdown_connections 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 -- 172.21.15.47:0/829043494 wait complete. 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 Processor -- start 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 -- start start 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 0x7f455c0fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.709+0000 7f4560898700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f455c107300 con 0x7f455c1013c0 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f455a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 0x7f455c0fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f455a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 0x7f455c0fe7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53046/0 (socket says 172.21.15.47:53046) 2024-04-25T20:49:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f455a59c700 1 -- 172.21.15.47:0/4050917488 learned_addr learned my addr 172.21.15.47:0/4050917488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:49:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f455a59c700 1 -- 172.21.15.47:0/4050917488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c0fed30 con 0x7f455c1013c0 2024-04-25T20:49:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f455a59c700 1 --2- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 0x7f455c0fe7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f454c005950 tx=0x7f454c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f454b7fe700 1 -- 172.21.15.47:0/4050917488 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f454c026070 con 0x7f455c1013c0 2024-04-25T20:49:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f454b7fe700 1 -- 172.21.15.47:0/4050917488 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c00bc00 con 0x7f455c1013c0 2024-04-25T20:49:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c0fcf20 con 0x7f455c1013c0 2024-04-25T20:49:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f454b7fe700 1 -- 172.21.15.47:0/4050917488 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f454c003a80 con 0x7f455c1013c0 2024-04-25T20:49:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.710+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c0fd460 con 0x7f455c1013c0 2024-04-25T20:49:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.711+0000 7f454b7fe700 1 -- 172.21.15.47:0/4050917488 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f454c029070 con 0x7f455c1013c0 2024-04-25T20:49:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.711+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c107a80 con 0x7f455c1013c0 2024-04-25T20:49:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.711+0000 7f454b7fe700 1 --2- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f45440396b0 0x7f454403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.711+0000 7f454b7fe700 1 -- 172.21.15.47:0/4050917488 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f454c01e070 con 0x7f455c1013c0 2024-04-25T20:49:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.711+0000 7f4559d9b700 1 --2- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f45440396b0 0x7f454403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.712+0000 7f4559d9b700 1 --2- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f45440396b0 0x7f454403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4550006fd0 tx=0x7f4550006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.717+0000 7f454b7fe700 1 -- 172.21.15.47:0/4050917488 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f454c013070 con 0x7f455c1013c0 2024-04-25T20:49:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.924+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f455c1910e0 con 0x7f455c1013c0 2024-04-25T20:49:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.927+0000 7f454b7fe700 1 -- 172.21.15.47:0/4050917488 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f454c030020 con 0x7f455c1013c0 2024-04-25T20:49:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f45440396b0 msgr2=0x7f454403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 --2- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f45440396b0 0x7f454403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4550006fd0 tx=0x7f4550006e40 comp rx=0 tx=0).stop 2024-04-25T20:49:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 msgr2=0x7f455c0fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 --2- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 0x7f455c0fe7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f454c005950 tx=0x7f454c0057f0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 shutdown_connections 2024-04-25T20:49:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 --2- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f45440396b0 0x7f454403bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 --2- 172.21.15.47:0/4050917488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c1013c0 0x7f455c0fe7f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 >> 172.21.15.47:0/4050917488 conn(0x7f455c0fc980 msgr2=0x7f455c06ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 shutdown_connections 2024-04-25T20:49:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:49:57.929+0000 7f4560898700 1 -- 172.21.15.47:0/4050917488 wait complete. 2024-04-25T20:49:58.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:58.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:58.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:49:58.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:58.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:58.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:49:58.562 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T20:49:58.562 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T20:49:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:49:59.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:58 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/4050917488' entity='client.admin' 2024-04-25T20:49:59.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:59.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:49:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:00.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 -- 172.21.15.47:0/4076360074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 msgr2=0x7fd124105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:00.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 --2- 172.21.15.47:0/4076360074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 0x7fd124105370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd114009910 tx=0x7fd114009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:00.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 -- 172.21.15.47:0/4076360074 shutdown_connections 2024-04-25T20:50:00.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 --2- 172.21.15.47:0/4076360074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 0x7fd124105370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 -- 172.21.15.47:0/4076360074 >> 172.21.15.47:0/4076360074 conn(0x7fd1240fc950 msgr2=0x7fd1240fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 -- 172.21.15.47:0/4076360074 shutdown_connections 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 -- 172.21.15.47:0/4076360074 wait complete. 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 Processor -- start 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.309+0000 7fd12a74a700 1 -- start start 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd12a74a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 0x7fd124197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd12a74a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd1241083d0 con 0x7fd124102f80 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd123fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 0x7fd124197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd123fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 0x7fd124197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53060/0 (socket says 172.21.15.47:53060) 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd123fff700 1 -- 172.21.15.47:0/17296562 learned_addr learned my addr 172.21.15.47:0/17296562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd123fff700 1 -- 172.21.15.47:0/17296562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd124197f80 con 0x7fd124102f80 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd123fff700 1 --2- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 0x7fd124197a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd114005b40 tx=0x7fd114004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd1217fa700 1 -- 172.21.15.47:0/17296562 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd114027070 con 0x7fd124102f80 2024-04-25T20:50:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd124198270 con 0x7fd124102f80 2024-04-25T20:50:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd1217fa700 1 -- 172.21.15.47:0/17296562 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd11400be10 con 0x7fd124102f80 2024-04-25T20:50:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12419af00 con 0x7fd124102f80 2024-04-25T20:50:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.310+0000 7fd1217fa700 1 -- 172.21.15.47:0/17296562 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd114022de0 con 0x7fd124102f80 2024-04-25T20:50:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.311+0000 7fd1217fa700 1 -- 172.21.15.47:0/17296562 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd11402a070 con 0x7fd124102f80 2024-04-25T20:50:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.311+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd124108b50 con 0x7fd124102f80 2024-04-25T20:50:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.311+0000 7fd1217fa700 1 --2- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd10c039650 0x7fd10c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.312+0000 7fd1217fa700 1 -- 172.21.15.47:0/17296562 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd11401e030 con 0x7fd124102f80 2024-04-25T20:50:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.312+0000 7fd1237fe700 1 --2- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd10c039650 0x7fd10c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.312+0000 7fd1237fe700 1 --2- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd10c039650 0x7fd10c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd118006fd0 tx=0x7fd118006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:00.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.316+0000 7fd1217fa700 1 -- 172.21.15.47:0/17296562 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd114013070 con 0x7fd124102f80 2024-04-25T20:50:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.528+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fd12402e1d0 con 0x7fd10c039650 2024-04-25T20:50:00.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.532+0000 7fd1217fa700 1 -- 172.21.15.47:0/17296562 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd12402e1d0 con 0x7fd10c039650 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd10c039650 msgr2=0x7fd10c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 --2- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd10c039650 0x7fd10c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd118006fd0 tx=0x7fd118006e40 comp rx=0 tx=0).stop 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 msgr2=0x7fd124197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 --2- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 0x7fd124197a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd114005b40 tx=0x7fd114004f80 comp rx=0 tx=0).stop 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 shutdown_connections 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 --2- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd10c039650 0x7fd10c03bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 --2- 172.21.15.47:0/17296562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd124102f80 0x7fd124197a40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 >> 172.21.15.47:0/17296562 conn(0x7fd1240fc950 msgr2=0x7fd1240fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 shutdown_connections 2024-04-25T20:50:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:00.534+0000 7fd12a74a700 1 -- 172.21.15.47:0/17296562 wait complete. 2024-04-25T20:50:01.201 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi092 2024-04-25T20:50:01.201 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:50:01.202 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.conf 2024-04-25T20:50:01.219 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:50:01.219 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:01.277 INFO:tasks.cephadm:Adding host smithi092 to orchestrator... 2024-04-25T20:50:01.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch host add smithi092 2024-04-25T20:50:01.523 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:01.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-25T20:50:01.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:01.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:01.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:02.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:02 smithi047 ceph-mon[29195]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:02.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:02 smithi047 ceph-mon[29195]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:02.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:02 smithi047 ceph-mon[29195]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:50:02.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.989+0000 7f67656d2700 1 -- 172.21.15.47:0/3823575474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 msgr2=0x7f67601088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:02.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.989+0000 7f67656d2700 1 --2- 172.21.15.47:0/3823575474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 0x7f67601088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6750009910 tx=0x7f6750009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:02.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.989+0000 7f67656d2700 1 -- 172.21.15.47:0/3823575474 shutdown_connections 2024-04-25T20:50:02.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.989+0000 7f67656d2700 1 --2- 172.21.15.47:0/3823575474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 0x7f67601088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:02.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.989+0000 7f67656d2700 1 -- 172.21.15.47:0/3823575474 >> 172.21.15.47:0/3823575474 conn(0x7f67600fdf60 msgr2=0x7f6760100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:02.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.989+0000 7f67656d2700 1 -- 172.21.15.47:0/3823575474 shutdown_connections 2024-04-25T20:50:02.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.989+0000 7f67656d2700 1 -- 172.21.15.47:0/3823575474 wait complete. 2024-04-25T20:50:02.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.989+0000 7f67656d2700 1 Processor -- start 2024-04-25T20:50:02.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.990+0000 7f67656d2700 1 -- start start 2024-04-25T20:50:02.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.990+0000 7f67656d2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 0x7f6760074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:02.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.990+0000 7f67656d2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f676010c940 con 0x7f67601084e0 2024-04-25T20:50:02.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.990+0000 7f675effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 0x7f6760074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:02.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.990+0000 7f675effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 0x7f6760074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53088/0 (socket says 172.21.15.47:53088) 2024-04-25T20:50:02.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.990+0000 7f675effd700 1 -- 172.21.15.47:0/3033843939 learned_addr learned my addr 172.21.15.47:0/3033843939 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:50:02.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.990+0000 7f675effd700 1 -- 172.21.15.47:0/3033843939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6760076c30 con 0x7f67601084e0 2024-04-25T20:50:02.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.990+0000 7f675effd700 1 --2- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 0x7f6760074d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67500056d0 tx=0x7f67500057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:02.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.991+0000 7f6747fff700 1 -- 172.21.15.47:0/3033843939 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6750026070 con 0x7f67601084e0 2024-04-25T20:50:02.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.991+0000 7f6747fff700 1 -- 172.21.15.47:0/3033843939 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f675000bc00 con 0x7f67601084e0 2024-04-25T20:50:02.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.991+0000 7f6747fff700 1 -- 172.21.15.47:0/3033843939 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6750003920 con 0x7f67601084e0 2024-04-25T20:50:02.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.991+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67600752e0 con 0x7f67601084e0 2024-04-25T20:50:02.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.991+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67600757a0 con 0x7f67601084e0 2024-04-25T20:50:02.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.991+0000 7f6747fff700 1 -- 172.21.15.47:0/3033843939 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6750029070 con 0x7f67601084e0 2024-04-25T20:50:02.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.992+0000 7f6747fff700 1 --2- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6748039650 0x7f674803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:02.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.992+0000 7f6747fff700 1 -- 172.21.15.47:0/3033843939 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f675001e070 con 0x7f67601084e0 2024-04-25T20:50:02.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.992+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6760103b90 con 0x7f67601084e0 2024-04-25T20:50:02.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.992+0000 7f675e7fc700 1 --2- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6748039650 0x7f674803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:02.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.992+0000 7f675e7fc700 1 --2- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6748039650 0x7f674803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6754006fd0 tx=0x7f6754006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:02.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:02.997+0000 7f6747fff700 1 -- 172.21.15.47:0/3033843939 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6750030070 con 0x7f67601084e0 2024-04-25T20:50:03.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:03.208+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}) -- 0x7f6760076270 con 0x7f6748039650 2024-04-25T20:50:04.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:03 smithi047 ceph-mon[29195]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:05.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:04 smithi047 ceph-mon[29195]: Deploying cephadm binary to smithi092 2024-04-25T20:50:07.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:06 smithi047 ceph-mon[29195]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:07.505 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi092' with addr '172.21.15.92' 2024-04-25T20:50:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.503+0000 7f6747fff700 1 -- 172.21.15.47:0/3033843939 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6760076270 con 0x7f6748039650 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6748039650 msgr2=0x7f674803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 --2- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6748039650 0x7f674803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6754006fd0 tx=0x7f6754006e40 comp rx=0 tx=0).stop 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 msgr2=0x7f6760074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 --2- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 0x7f6760074d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67500056d0 tx=0x7f67500057b0 comp rx=0 tx=0).stop 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 shutdown_connections 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 --2- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6748039650 0x7f674803bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 --2- 172.21.15.47:0/3033843939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67601084e0 0x7f6760074d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 >> 172.21.15.47:0/3033843939 conn(0x7f67600fdf60 msgr2=0x7f67600fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 shutdown_connections 2024-04-25T20:50:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:07.506+0000 7f67656d2700 1 -- 172.21.15.47:0/3033843939 wait complete. 2024-04-25T20:50:08.118 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T20:50:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:08.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:08 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:08.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:08 smithi047 ceph-mon[29195]: Added host smithi092 2024-04-25T20:50:08.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:08 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:08.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:08 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:09.650 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:09 smithi047 ceph-mon[29195]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:09.650 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:09 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.755+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/3057005340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 msgr2=0x7f7c441088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.755+0000 7f7c4b6dc700 1 --2- 172.21.15.47:0/3057005340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 0x7f7c441088c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009910 tx=0x7f7c40009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.755+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/3057005340 shutdown_connections 2024-04-25T20:50:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.755+0000 7f7c4b6dc700 1 --2- 172.21.15.47:0/3057005340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 0x7f7c441088c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.755+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/3057005340 >> 172.21.15.47:0/3057005340 conn(0x7f7c440fdf80 msgr2=0x7f7c441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.755+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/3057005340 shutdown_connections 2024-04-25T20:50:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.755+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/3057005340 wait complete. 2024-04-25T20:50:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.756+0000 7f7c4b6dc700 1 Processor -- start 2024-04-25T20:50:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.756+0000 7f7c4b6dc700 1 -- start start 2024-04-25T20:50:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.756+0000 7f7c4b6dc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 0x7f7c44197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.756+0000 7f7c4b6dc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410c940 con 0x7f7c441084e0 2024-04-25T20:50:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.756+0000 7f7c49478700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 0x7f7c44197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.756+0000 7f7c49478700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 0x7f7c44197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55286/0 (socket says 172.21.15.47:55286) 2024-04-25T20:50:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.756+0000 7f7c49478700 1 -- 172.21.15.47:0/2623738170 learned_addr learned my addr 172.21.15.47:0/2623738170 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:50:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.757+0000 7f7c49478700 1 -- 172.21.15.47:0/2623738170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c441983f0 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.757+0000 7f7c49478700 1 --2- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 0x7f7c44197eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c40005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.757+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/2623738170 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40026070 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.757+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/2623738170 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4000bba0 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.757+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c441986e0 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.757+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/2623738170 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40003910 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.757+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4419c780 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.758+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/2623738170 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7c40029070 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.758+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c44103b90 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.758+0000 7f7c3a7fc700 1 --2- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7c300396b0 0x7f7c3003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.758+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/2623738170 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7c4001e070 con 0x7f7c441084e0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.758+0000 7f7c48c77700 1 --2- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7c300396b0 0x7f7c3003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.759+0000 7f7c48c77700 1 --2- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7c300396b0 0x7f7c3003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7c34006fd0 tx=0x7f7c34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:09.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.763+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/2623738170 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c40013070 con 0x7f7c441084e0 2024-04-25T20:50:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.975+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c44199210 con 0x7f7c300396b0 2024-04-25T20:50:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.976+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/2623738170 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f7c44199210 con 0x7f7c300396b0 2024-04-25T20:50:09.977 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:50:09.977 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}] 2024-04-25T20:50:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7c300396b0 msgr2=0x7f7c3003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 --2- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7c300396b0 0x7f7c3003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7c34006fd0 tx=0x7f7c34006e40 comp rx=0 tx=0).stop 2024-04-25T20:50:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 msgr2=0x7f7c44197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 --2- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 0x7f7c44197eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c40005780 comp rx=0 tx=0).stop 2024-04-25T20:50:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 shutdown_connections 2024-04-25T20:50:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 --2- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7c300396b0 0x7f7c3003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 --2- 172.21.15.47:0/2623738170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441084e0 0x7f7c44197eb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 >> 172.21.15.47:0/2623738170 conn(0x7f7c440fdf80 msgr2=0x7f7c4410a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 shutdown_connections 2024-04-25T20:50:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:09.979+0000 7f7c4b6dc700 1 -- 172.21.15.47:0/2623738170 wait complete. 2024-04-25T20:50:10.431 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2024-04-25T20:50:10.431 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:50:10.431 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2024-04-25T20:50:10.450 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:50:10.450 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:10.507 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2024-04-25T20:50:10.507 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch host add smithi186 2024-04-25T20:50:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:11.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:11 smithi047 ceph-mon[29195]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:11.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:11 smithi047 ceph-mon[29195]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:50:11.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.070+0000 7f4b17490700 1 -- 172.21.15.47:0/2066300284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 msgr2=0x7f4b101039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.070+0000 7f4b17490700 1 --2- 172.21.15.47:0/2066300284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 0x7f4b101039f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009910 tx=0x7f4b00009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.070+0000 7f4b17490700 1 -- 172.21.15.47:0/2066300284 shutdown_connections 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.070+0000 7f4b17490700 1 --2- 172.21.15.47:0/2066300284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 0x7f4b101039f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.070+0000 7f4b17490700 1 -- 172.21.15.47:0/2066300284 >> 172.21.15.47:0/2066300284 conn(0x7f4b100fce40 msgr2=0x7f4b100ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.071+0000 7f4b17490700 1 -- 172.21.15.47:0/2066300284 shutdown_connections 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.071+0000 7f4b17490700 1 -- 172.21.15.47:0/2066300284 wait complete. 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.071+0000 7f4b17490700 1 Processor -- start 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.071+0000 7f4b17490700 1 -- start start 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.071+0000 7f4b17490700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 0x7f4b10101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.071+0000 7f4b17490700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010c9b0 con 0x7f4b10103610 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.071+0000 7f4b1522c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 0x7f4b10101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b1522c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 0x7f4b10101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55300/0 (socket says 172.21.15.47:55300) 2024-04-25T20:50:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b1522c700 1 -- 172.21.15.47:0/1093990079 learned_addr learned my addr 172.21.15.47:0/1093990079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:50:12.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b1522c700 1 -- 172.21.15.47:0/1093990079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b10103110 con 0x7f4b10103610 2024-04-25T20:50:12.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b1522c700 1 --2- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 0x7f4b10101270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b00006010 tx=0x7f4b00005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b067fc700 1 -- 172.21.15.47:0/1093990079 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b00026070 con 0x7f4b10103610 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b067fc700 1 -- 172.21.15.47:0/1093990079 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0000bc00 con 0x7f4b10103610 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b101017b0 con 0x7f4b10103610 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b067fc700 1 -- 172.21.15.47:0/1093990079 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b00003910 con 0x7f4b10103610 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.072+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b10101c70 con 0x7f4b10103610 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.073+0000 7f4b067fc700 1 -- 172.21.15.47:0/1093990079 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4b00029070 con 0x7f4b10103610 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.073+0000 7f4b067fc700 1 --2- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4afc0396b0 0x7f4afc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.073+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1006b190 con 0x7f4b10103610 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.073+0000 7f4b067fc700 1 -- 172.21.15.47:0/1093990079 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4b0001e070 con 0x7f4b10103610 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.073+0000 7f4b14a2b700 1 --2- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4afc0396b0 0x7f4afc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:12.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.074+0000 7f4b14a2b700 1 --2- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4afc0396b0 0x7f4afc03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c006fd0 tx=0x7f4b0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.078+0000 7f4b067fc700 1 -- 172.21.15.47:0/1093990079 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b00030020 con 0x7f4b10103610 2024-04-25T20:50:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:12.290+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) -- 0x7f4b101027c0 con 0x7f4afc0396b0 2024-04-25T20:50:13.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:13.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:13.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:13.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:13.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:13.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:13.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:13.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:13.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:13 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:14.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:14 smithi047 ceph-mon[29195]: Deploying cephadm binary to smithi186 2024-04-25T20:50:14.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:14 smithi047 ceph-mon[29195]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T20:50:14.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:14 smithi047 ceph-mon[29195]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:15.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:15 smithi047 ceph-mon[29195]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.509+0000 7f4b067fc700 1 -- 172.21.15.47:0/1093990079 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4b101027c0 con 0x7f4afc0396b0 2024-04-25T20:50:16.510 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi186' with addr '172.21.15.186' 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4afc0396b0 msgr2=0x7f4afc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 --2- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4afc0396b0 0x7f4afc03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c006fd0 tx=0x7f4b0c006e40 comp rx=0 tx=0).stop 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 msgr2=0x7f4b10101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 --2- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 0x7f4b10101270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b00006010 tx=0x7f4b00005780 comp rx=0 tx=0).stop 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 shutdown_connections 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 --2- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4afc0396b0 0x7f4afc03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 --2- 172.21.15.47:0/1093990079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b10103610 0x7f4b10101270 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 >> 172.21.15.47:0/1093990079 conn(0x7f4b100fce40 msgr2=0x7f4b101043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 shutdown_connections 2024-04-25T20:50:16.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:16.511+0000 7f4b17490700 1 -- 172.21.15.47:0/1093990079 wait complete. 2024-04-25T20:50:17.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T20:50:17.350 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:17.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:17.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:17.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:50:17.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:17.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:17.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:17.744 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:17.744 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: Added host smithi186 2024-04-25T20:50:17.744 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:17.744 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:18.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.011+0000 7fae83d49700 1 -- 172.21.15.47:0/3652566732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 msgr2=0x7fae7c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.011+0000 7fae83d49700 1 --2- 172.21.15.47:0/3652566732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 0x7fae7c1088c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae78009910 tx=0x7fae78009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.011+0000 7fae83d49700 1 -- 172.21.15.47:0/3652566732 shutdown_connections 2024-04-25T20:50:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.011+0000 7fae83d49700 1 --2- 172.21.15.47:0/3652566732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 0x7fae7c1088c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.011+0000 7fae83d49700 1 -- 172.21.15.47:0/3652566732 >> 172.21.15.47:0/3652566732 conn(0x7fae7c0fdf80 msgr2=0x7fae7c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:19.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.011+0000 7fae83d49700 1 -- 172.21.15.47:0/3652566732 shutdown_connections 2024-04-25T20:50:19.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.011+0000 7fae83d49700 1 -- 172.21.15.47:0/3652566732 wait complete. 2024-04-25T20:50:19.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.011+0000 7fae83d49700 1 Processor -- start 2024-04-25T20:50:19.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae83d49700 1 -- start start 2024-04-25T20:50:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae83d49700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 0x7fae7c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae83d49700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c10c940 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae81ae5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 0x7fae7c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae81ae5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 0x7fae7c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43658/0 (socket says 172.21.15.47:43658) 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae81ae5700 1 -- 172.21.15.47:0/2753147976 learned_addr learned my addr 172.21.15.47:0/2753147976 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae81ae5700 1 -- 172.21.15.47:0/2753147976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c1983f0 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae81ae5700 1 --2- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 0x7fae7c197eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae780056d0 tx=0x7fae780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae72ffd700 1 -- 172.21.15.47:0/2753147976 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae78026070 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c1986e0 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae72ffd700 1 -- 172.21.15.47:0/2753147976 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7800bba0 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.012+0000 7fae72ffd700 1 -- 172.21.15.47:0/2753147976 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae78003920 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.013+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c19c780 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.013+0000 7fae72ffd700 1 -- 172.21.15.47:0/2753147976 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fae78029070 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.013+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c103b90 con 0x7fae7c1084e0 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.014+0000 7fae72ffd700 1 --2- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fae68039650 0x7fae6803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.014+0000 7fae72ffd700 1 -- 172.21.15.47:0/2753147976 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fae7801e070 con 0x7fae7c1084e0 2024-04-25T20:50:19.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.014+0000 7fae812e4700 1 --2- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fae68039650 0x7fae6803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:19.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.014+0000 7fae812e4700 1 --2- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fae68039650 0x7fae6803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fae6c006fd0 tx=0x7fae6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:19.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.019+0000 7fae72ffd700 1 -- 172.21.15.47:0/2753147976 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae78013070 con 0x7fae7c1084e0 2024-04-25T20:50:19.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.234+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae7c199210 con 0x7fae68039650 2024-04-25T20:50:19.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.235+0000 7fae72ffd700 1 -- 172.21.15.47:0/2753147976 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fae7c199210 con 0x7fae68039650 2024-04-25T20:50:19.236 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:50:19.236 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2024-04-25T20:50:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fae68039650 msgr2=0x7fae6803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 --2- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fae68039650 0x7fae6803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fae6c006fd0 tx=0x7fae6c006e40 comp rx=0 tx=0).stop 2024-04-25T20:50:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 msgr2=0x7fae7c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 --2- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 0x7fae7c197eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae780056d0 tx=0x7fae780057b0 comp rx=0 tx=0).stop 2024-04-25T20:50:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 shutdown_connections 2024-04-25T20:50:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 --2- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fae68039650 0x7fae6803bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 --2- 172.21.15.47:0/2753147976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae7c1084e0 0x7fae7c197eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 >> 172.21.15.47:0/2753147976 conn(0x7fae7c0fdf80 msgr2=0x7fae7c10a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 shutdown_connections 2024-04-25T20:50:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:19.237+0000 7fae83d49700 1 -- 172.21.15.47:0/2753147976 wait complete. 2024-04-25T20:50:19.772 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T20:50:19.773 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T20:50:19.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:19 smithi047 ceph-mon[29195]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:19.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:20.017 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:20.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:20 smithi047 ceph-mon[29195]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:50:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.412+0000 7f75fe353700 1 -- 172.21.15.47:0/3933778282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 msgr2=0x7f75f8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.412+0000 7f75fe353700 1 --2- 172.21.15.47:0/3933778282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 0x7f75f8108930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009910 tx=0x7f75e8009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.412+0000 7f75fe353700 1 -- 172.21.15.47:0/3933778282 shutdown_connections 2024-04-25T20:50:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.412+0000 7f75fe353700 1 --2- 172.21.15.47:0/3933778282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 0x7f75f8108930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.412+0000 7f75fe353700 1 -- 172.21.15.47:0/3933778282 >> 172.21.15.47:0/3933778282 conn(0x7f75f8077ac0 msgr2=0x7f75f8077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.412+0000 7f75fe353700 1 -- 172.21.15.47:0/3933778282 shutdown_connections 2024-04-25T20:50:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.412+0000 7f75fe353700 1 -- 172.21.15.47:0/3933778282 wait complete. 2024-04-25T20:50:21.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.413+0000 7f75fe353700 1 Processor -- start 2024-04-25T20:50:21.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.413+0000 7f75fe353700 1 -- start start 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75fe353700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 0x7f75f8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75fe353700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810c9b0 con 0x7f75f8108550 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75f7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 0x7f75f8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75f7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 0x7f75f8197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43678/0 (socket says 172.21.15.47:43678) 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75f7fff700 1 -- 172.21.15.47:0/1890546780 learned_addr learned my addr 172.21.15.47:0/1890546780 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75f7fff700 1 -- 172.21.15.47:0/1890546780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f8198370 con 0x7f75f8108550 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75f7fff700 1 --2- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 0x7f75f8197e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75e80056d0 tx=0x7f75e80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75f57fa700 1 -- 172.21.15.47:0/1890546780 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75e8026070 con 0x7f75f8108550 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75f57fa700 1 -- 172.21.15.47:0/1890546780 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75e800bba0 con 0x7f75f8108550 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f8198660 con 0x7f75f8108550 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75f57fa700 1 -- 172.21.15.47:0/1890546780 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75e8003920 con 0x7f75f8108550 2024-04-25T20:50:21.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.414+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f819c700 con 0x7f75f8108550 2024-04-25T20:50:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.415+0000 7f75f57fa700 1 -- 172.21.15.47:0/1890546780 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f75e8029070 con 0x7f75f8108550 2024-04-25T20:50:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.415+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f8103cf0 con 0x7f75f8108550 2024-04-25T20:50:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.415+0000 7f75f57fa700 1 --2- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f75e4039650 0x7f75e403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.416+0000 7f75f57fa700 1 -- 172.21.15.47:0/1890546780 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f75e801e070 con 0x7f75f8108550 2024-04-25T20:50:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.416+0000 7f75f77fe700 1 --2- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f75e4039650 0x7f75e403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.416+0000 7f75f77fe700 1 --2- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f75e4039650 0x7f75e403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f75e0006fd0 tx=0x7f75e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.421+0000 7f75f57fa700 1 -- 172.21.15.47:0/1890546780 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75e8013070 con 0x7f75f8108550 2024-04-25T20:50:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:21.638+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f75f8199190 con 0x7f75f8108550 2024-04-25T20:50:21.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:21 smithi047 ceph-mon[29195]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:22.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.572+0000 7f75f57fa700 1 -- 172.21.15.47:0/1890546780 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f75e8030090 con 0x7f75f8108550 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f75e4039650 msgr2=0x7f75e403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 --2- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f75e4039650 0x7f75e403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f75e0006fd0 tx=0x7f75e0006e40 comp rx=0 tx=0).stop 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 msgr2=0x7f75f8197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 --2- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 0x7f75f8197e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75e80056d0 tx=0x7f75e80057b0 comp rx=0 tx=0).stop 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 shutdown_connections 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 --2- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f75e4039650 0x7f75e403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 --2- 172.21.15.47:0/1890546780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75f8108550 0x7f75f8197e30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 >> 172.21.15.47:0/1890546780 conn(0x7f75f8077ac0 msgr2=0x7f75f8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 shutdown_connections 2024-04-25T20:50:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:22.575+0000 7f75fe353700 1 -- 172.21.15.47:0/1890546780 wait complete. 2024-04-25T20:50:22.577 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-04-25T20:50:22.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:22 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1890546780' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T20:50:22.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:22.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:22.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:22.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:22.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:22.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:22.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:23.225 INFO:tasks.cephadm:Adding mon.a on smithi047 2024-04-25T20:50:23.225 INFO:tasks.cephadm:Adding mon.b on smithi092 2024-04-25T20:50:23.225 INFO:tasks.cephadm:Adding mon.c on smithi186 2024-04-25T20:50:23.226 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c' 2024-04-25T20:50:23.448 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:23.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:23.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T20:50:23.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:23.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:23.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1890546780' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T20:50:23.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:23.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:23.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:23.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.775+0000 7f6198755700 1 -- 172.21.15.186:0/3742032319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 msgr2=0x7f6190108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.775+0000 7f6198755700 1 --2- 172.21.15.186:0/3742032319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 0x7f6190108810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f618c009910 tx=0x7f618c009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.775+0000 7f6198755700 1 -- 172.21.15.186:0/3742032319 shutdown_connections 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.775+0000 7f6198755700 1 --2- 172.21.15.186:0/3742032319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 0x7f6190108810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.775+0000 7f6198755700 1 -- 172.21.15.186:0/3742032319 >> 172.21.15.186:0/3742032319 conn(0x7f61900788d0 msgr2=0x7f6190078ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.775+0000 7f6198755700 1 -- 172.21.15.186:0/3742032319 shutdown_connections 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.775+0000 7f6198755700 1 -- 172.21.15.186:0/3742032319 wait complete. 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.776+0000 7f6198755700 1 Processor -- start 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.776+0000 7f6198755700 1 -- start start 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.776+0000 7f6198755700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 0x7f6190197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:24.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.776+0000 7f6198755700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f619010c920 con 0x7f6190108430 2024-04-25T20:50:24.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.776+0000 7f61964f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 0x7f6190197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:24.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f61964f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 0x7f6190197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.186:53492/0 (socket says 172.21.15.186:53492) 2024-04-25T20:50:24.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f61964f1700 1 -- 172.21.15.186:0/3807456529 learned_addr learned my addr 172.21.15.186:0/3807456529 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:50:24.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f61964f1700 1 -- 172.21.15.186:0/3807456529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61901983d0 con 0x7f6190108430 2024-04-25T20:50:24.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f61964f1700 1 --2- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 0x7f6190197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f618c0056d0 tx=0x7f618c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:24.780 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f61877fe700 1 -- 172.21.15.186:0/3807456529 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c026070 con 0x7f6190108430 2024-04-25T20:50:24.780 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61901986c0 con 0x7f6190108430 2024-04-25T20:50:24.780 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f61877fe700 1 -- 172.21.15.186:0/3807456529 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c00bba0 con 0x7f6190108430 2024-04-25T20:50:24.780 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f61877fe700 1 -- 172.21.15.186:0/3807456529 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c003920 con 0x7f6190108430 2024-04-25T20:50:24.780 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.777+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619019c760 con 0x7f6190108430 2024-04-25T20:50:24.780 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.778+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619006a620 con 0x7f6190108430 2024-04-25T20:50:24.780 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.778+0000 7f61877fe700 1 -- 172.21.15.186:0/3807456529 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f618c029070 con 0x7f6190108430 2024-04-25T20:50:24.781 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.778+0000 7f61877fe700 1 --2- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f617c039650 0x7f617c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:24.781 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.778+0000 7f61877fe700 1 -- 172.21.15.186:0/3807456529 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f618c01e070 con 0x7f6190108430 2024-04-25T20:50:24.781 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.779+0000 7f6195cf0700 1 --2- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f617c039650 0x7f617c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:24.781 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.779+0000 7f6195cf0700 1 --2- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f617c039650 0x7f617c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6180006fd0 tx=0x7f6180006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:24.785 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.784+0000 7f61877fe700 1 -- 172.21.15.186:0/3807456529 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f618c013070 con 0x7f6190108430 2024-04-25T20:50:24.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:24 smithi047 ceph-mon[29195]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:50:24.994 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:24.993+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}) -- 0x7f61901991f0 con 0x7f617c039650 2024-04-25T20:50:25.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.000+0000 7f61877fe700 1 -- 172.21.15.186:0/3807456529 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f61901991f0 con 0x7f617c039650 2024-04-25T20:50:25.001 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f617c039650 msgr2=0x7f617c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 --2- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f617c039650 0x7f617c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6180006fd0 tx=0x7f6180006e40 comp rx=0 tx=0).stop 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 msgr2=0x7f6190197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 --2- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 0x7f6190197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f618c0056d0 tx=0x7f618c0057b0 comp rx=0 tx=0).stop 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 shutdown_connections 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 --2- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f617c039650 0x7f617c03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 --2- 172.21.15.186:0/3807456529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6190108430 0x7f6190197e90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 >> 172.21.15.186:0/3807456529 conn(0x7f61900788d0 msgr2=0x7f61900fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:25.003 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 shutdown_connections 2024-04-25T20:50:25.004 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:25.002+0000 7f6198755700 1 -- 172.21.15.186:0/3807456529 wait complete. 2024-04-25T20:50:25.595 DEBUG:teuthology.orchestra.run.smithi092:mon.b> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T20:50:25.598 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T20:50:25.600 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:50:25.601 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:50:25.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:25 smithi047 ceph-mon[29195]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:25.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:25.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:25.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:25.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:25.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:25.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:25.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:26.642 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:26.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:26 smithi047 ceph-mon[29195]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:26.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:26 smithi047 ceph-mon[29195]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c;count:3 2024-04-25T20:50:26.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:26 smithi047 ceph-mon[29195]: Deploying daemon mon.c on smithi186 2024-04-25T20:50:27.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:27 smithi047 ceph-mon[29195]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.210 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 podman[32482]: 2024-04-25 20:50:29.209144626 +0000 UTC m=+0.319578781 container create 7220ee8fe3f6109ae1d799d50819919eb4eaadf654baa479d17d8267b1fefccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-c, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T20:50:29.491 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 -- 172.21.15.186:0/3443845896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 msgr2=0x7f9e90101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:29.491 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 --2- 172.21.15.186:0/3443845896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 0x7f9e90101990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800b0a0 tx=0x7f9e8800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 -- 172.21.15.186:0/3443845896 shutdown_connections 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 --2- 172.21.15.186:0/3443845896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 0x7f9e90101990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 -- 172.21.15.186:0/3443845896 >> 172.21.15.186:0/3443845896 conn(0x7f9e900fce60 msgr2=0x7f9e900ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 -- 172.21.15.186:0/3443845896 shutdown_connections 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 -- 172.21.15.186:0/3443845896 wait complete. 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 Processor -- start 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.490+0000 7f9e96a87700 1 -- start start 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.491+0000 7f9e96a87700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 0x7f9e90196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:29.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.491+0000 7f9e96a87700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010b830 con 0x7f9e901015b0 2024-04-25T20:50:29.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.491+0000 7f9e94823700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 0x7f9e90196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:29.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.491+0000 7f9e94823700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 0x7f9e90196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.186:53520/0 (socket says 172.21.15.186:53520) 2024-04-25T20:50:29.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.491+0000 7f9e94823700 1 -- 172.21.15.186:0/1224724809 learned_addr learned my addr 172.21.15.186:0/1224724809 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:50:29.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.491+0000 7f9e94823700 1 -- 172.21.15.186:0/1224724809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e90197270 con 0x7f9e901015b0 2024-04-25T20:50:29.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.491+0000 7f9e94823700 1 --2- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 0x7f9e90196d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e88006130 tx=0x7f9e880097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:29.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.492+0000 7f9e8dffb700 1 -- 172.21.15.186:0/1224724809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e88021020 con 0x7f9e901015b0 2024-04-25T20:50:29.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.492+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e90197500 con 0x7f9e901015b0 2024-04-25T20:50:29.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.492+0000 7f9e8dffb700 1 -- 172.21.15.186:0/1224724809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e880079b0 con 0x7f9e901015b0 2024-04-25T20:50:29.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.492+0000 7f9e8dffb700 1 -- 172.21.15.186:0/1224724809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e88017040 con 0x7f9e901015b0 2024-04-25T20:50:29.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.492+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9019b600 con 0x7f9e901015b0 2024-04-25T20:50:29.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.493+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e90102a90 con 0x7f9e901015b0 2024-04-25T20:50:29.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.493+0000 7f9e8dffb700 1 -- 172.21.15.186:0/1224724809 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9e88024020 con 0x7f9e901015b0 2024-04-25T20:50:29.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.493+0000 7f9e8dffb700 1 --2- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9e7c039650 0x7f9e7c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:29.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.493+0000 7f9e8dffb700 1 -- 172.21.15.186:0/1224724809 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9e8800f050 con 0x7f9e901015b0 2024-04-25T20:50:29.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.493+0000 7f9e8ffff700 1 --2- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9e7c039650 0x7f9e7c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:29.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.494+0000 7f9e8ffff700 1 --2- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9e7c039650 0x7f9e7c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9e80006fd0 tx=0x7f9e80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:29.499 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.499+0000 7f9e8dffb700 1 -- 172.21.15.186:0/1224724809 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e8802b020 con 0x7f9e901015b0 2024-04-25T20:50:29.613 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.612+0000 7f9e8dffb700 1 -- 172.21.15.186:0/1224724809 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9e8802ebe0 con 0x7f9e901015b0 2024-04-25T20:50:29.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 podman[32482]: 2024-04-25 20:50:29.554477845 +0000 UTC m=+0.664911998 container init 7220ee8fe3f6109ae1d799d50819919eb4eaadf654baa479d17d8267b1fefccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T20:50:29.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 podman[32482]: 2024-04-25 20:50:29.557737011 +0000 UTC m=+0.668171175 container start 7220ee8fe3f6109ae1d799d50819919eb4eaadf654baa479d17d8267b1fefccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T20:50:29.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T20:50:29.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pidfile_write: ignore empty --pid-file 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: load: jerasure load: lrc 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: RocksDB version: 7.9.2 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Git sha 0 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: DB SUMMARY 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: DB Session ID: QHHJ74G07Q67RMKLS5A8 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: CURRENT file: CURRENT 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T20:50:29.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.error_if_exists: 0 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.create_if_missing: 0 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.paranoid_checks: 1 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.env: 0x55bbd4b17cc0 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.info_log: 0x55bbd67d1be0 2024-04-25T20:50:29.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T20:50:29.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.statistics: (nil) 2024-04-25T20:50:29.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.use_fsync: 0 2024-04-25T20:50:29.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_log_file_size: 0 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.allow_fallocate: 1 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.use_direct_reads: 0 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.db_log_dir: 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.wal_dir: 2024-04-25T20:50:29.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.write_buffer_manager: 0x55bbd67650e0 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T20:50:29.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.unordered_write: 0 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.row_cache: None 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.wal_filter: None 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T20:50:29.727 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.two_write_queues: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.wal_compression: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.atomic_flush: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.log_readahead_size: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T20:50:29.728 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_background_jobs: 2 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_background_compactions: -1 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_subcompactions: 1 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_open_files: -1 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T20:50:29.729 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T20:50:29.730 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T20:50:29.730 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T20:50:29.730 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_background_flushes: -1 2024-04-25T20:50:29.730 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Compression algorithms supported: 2024-04-25T20:50:29.730 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T20:50:29.730 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: kZSTD supported: 0 2024-04-25T20:50:29.730 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: kXpressCompression supported: 0 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: kZlibCompression supported: 1 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: kSnappyCompression supported: 1 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: kLZ4Compression supported: 1 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: kBZip2Compression supported: 0 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T20:50:29.732 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.merge_operator: 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_filter: None 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bbd67d1dc0) 2024-04-25T20:50:29.733 INFO:journalctl@ceph.mon.c.smithi186.stdout: cache_index_and_filter_blocks: 1 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: pin_top_level_index_and_filter: 1 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_type: 0 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: data_block_index_type: 0 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_shortening: 1 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: checksum: 4 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: no_block_cache: 0 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache: 0x55bbd67971f0 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_name: BinnedLRUCache 2024-04-25T20:50:29.734 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_options: 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: capacity : 536870912 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: num_shard_bits : 4 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: strict_capacity_limit : 0 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: high_pri_pool_ratio: 0.000 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_compressed: (nil) 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: persistent_cache: (nil) 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_size: 4096 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_size_deviation: 10 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_restart_interval: 16 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_block_restart_interval: 1 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: metadata_block_size: 4096 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: partition_filters: 0 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: use_delta_encoding: 1 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: filter_policy: bloomfilter 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: whole_key_filtering: 1 2024-04-25T20:50:29.735 INFO:journalctl@ceph.mon.c.smithi186.stdout: verify_compression: 0 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout: read_amp_bytes_per_bit: 0 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout: format_version: 5 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_index_compression: 1 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_align: 0 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout: max_auto_readahead_size: 262144 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout: prepopulate_block_cache: 0 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout: initial_auto_readahead_size: 8192 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T20:50:29.736 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression: NoCompression 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.num_levels: 7 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:50:29.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T20:50:29.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T20:50:29.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T20:50:29.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T20:50:29.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T20:50:29.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T20:50:29.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T20:50:29.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T20:50:29.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T20:50:29.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T20:50:29.740 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T20:50:29.740 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T20:50:29.740 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T20:50:29.740 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T20:50:29.741 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.inplace_update_support: 0 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.bloom_locality: 0 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.max_successive_merges: 0 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T20:50:29.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.ttl: 2592000 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.enable_blob_files: false 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.min_blob_size: 0 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T20:50:29.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c9b07bff-24e1-4d9c-8674-54b41f5225c4 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078229587991, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078229588571, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714078229, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c9b07bff-24e1-4d9c-8674-54b41f5225c4", "db_session_id": "QHHJ74G07Q67RMKLS5A8", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078229588677, "job": 1, "event": "recovery_finished"} 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bbd68ac000 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: DB pointer 0x55bbd6896000 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T20:50:29.744 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** DB Stats ** 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T20:50:29.745 INFO:journalctl@ceph.mon.c.smithi186.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-25T20:50:29.746 INFO:journalctl@ceph.mon.c.smithi186.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: Block cache BinnedLRUCache@0x55bbd67971f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-25T20:50:29.747 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: using public_addr v2:172.21.15.186:0/0 -> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: starting mon.c rank -1 at public addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] at bind addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(???) e0 preinit fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout: e1 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout: legacy client fscid: -1 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout: No filesystems configured 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T20:50:29.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:29.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Deploying cephadm binary to smithi186 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Added host smithi186 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1890546780' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1890546780' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c;count:3 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: Deploying daemon mon.c on smithi186 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 ceph-mon[32529]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 bash[32482]: 7220ee8fe3f6109ae1d799d50819919eb4eaadf654baa479d17d8267b1fefccb 2024-04-25T20:50:29.753 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:29 smithi186 systemd[1]: Started Ceph mon.c for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:50:29.793 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:29.791+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9e900640a0 con 0x7f9e901015b0 2024-04-25T20:50:29.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:29 smithi047 ceph-mon[29195]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:34.626 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.625+0000 7f9e8dffb700 1 -- 172.21.15.186:0/1224724809 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f9e88003f80 con 0x7f9e901015b0 2024-04-25T20:50:34.627 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:50:34.627 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","modified":"2024-04-25T20:50:29.613167Z","created":"2024-04-25T20:48:48.454006Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T20:50:34.629 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.628+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9e7c039650 msgr2=0x7f9e7c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.628+0000 7f9e96a87700 1 --2- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9e7c039650 0x7f9e7c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9e80006fd0 tx=0x7f9e80006e40 comp rx=0 tx=0).stop 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.628+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 msgr2=0x7f9e90196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.628+0000 7f9e96a87700 1 --2- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 0x7f9e90196d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e88006130 tx=0x7f9e880097d0 comp rx=0 tx=0).stop 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.628+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 shutdown_connections 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.628+0000 7f9e96a87700 1 --2- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9e7c039650 0x7f9e7c03bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.628+0000 7f9e96a87700 1 --2- 172.21.15.186:0/1224724809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e901015b0 0x7f9e90196d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.629+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 >> 172.21.15.186:0/1224724809 conn(0x7f9e900fce60 msgr2=0x7f9e90102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.629+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 shutdown_connections 2024-04-25T20:50:34.630 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:34.629+0000 7f9e96a87700 1 -- 172.21.15.186:0/1224724809 wait complete. 2024-04-25T20:50:34.631 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: mon.a calling monitor election 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/1224724809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: mon.c calling monitor election 2024-04-25T20:50:34.950 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: monmap epoch 2 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: last_changed 2024-04-25T20:50:29.613167+0000 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: min_mon_release 19 (squid) 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: election_strategy: 1 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: fsmap 2024-04-25T20:50:34.951 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:34.952 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: mgrmap e13: x(active, since 48s) 2024-04-25T20:50:34.952 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: overall HEALTH_OK 2024-04-25T20:50:34.952 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:34.952 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:35.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:50:35.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: mon.a calling monitor election 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/1224724809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: mon.c calling monitor election 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:50:35.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: monmap epoch 2 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: last_changed 2024-04-25T20:50:29.613167+0000 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: election_strategy: 1 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: fsmap 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: mgrmap e13: x(active, since 48s) 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:35.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:35 smithi186 ceph-mon[32529]: Deploying daemon mon.b on smithi092 2024-04-25T20:50:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:36.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:36.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:36.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:36.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:35 smithi047 ceph-mon[29195]: Deploying daemon mon.b on smithi092 2024-04-25T20:50:36.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:36.229 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:50:36.229 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:50:36.475 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:50:36.896 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:36 smithi186 ceph-mon[32529]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:37.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:36 smithi047 ceph-mon[29195]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:37.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:50:36 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:50:36.612+0000 7f98465d7700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T20:50:37.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.834+0000 7f85f6188700 1 -- 172.21.15.186:0/4159948514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 msgr2=0x7f85f0103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:37.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.834+0000 7f85f6188700 1 --2- 172.21.15.186:0/4159948514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 0x7f85f0103ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009910 tx=0x7f85e0009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:37.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.834+0000 7f85f6188700 1 -- 172.21.15.186:0/4159948514 shutdown_connections 2024-04-25T20:50:37.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.834+0000 7f85f6188700 1 --2- 172.21.15.186:0/4159948514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 0x7f85f0103ad0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:37.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.834+0000 7f85f6188700 1 -- 172.21.15.186:0/4159948514 >> 172.21.15.186:0/4159948514 conn(0x7f85f00fcf20 msgr2=0x7f85f00ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:37.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.834+0000 7f85f6188700 1 -- 172.21.15.186:0/4159948514 shutdown_connections 2024-04-25T20:50:37.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.834+0000 7f85f6188700 1 -- 172.21.15.186:0/4159948514 wait complete. 2024-04-25T20:50:37.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.835+0000 7f85f6188700 1 Processor -- start 2024-04-25T20:50:37.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.835+0000 7f85f6188700 1 -- start start 2024-04-25T20:50:37.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.835+0000 7f85f6188700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 0x7f85f0101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:37.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.835+0000 7f85f6188700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 0x7f85f0101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:37.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.835+0000 7f85f6188700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010ca90 con 0x7f85f01036f0 2024-04-25T20:50:37.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.835+0000 7f85f6188700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010c910 con 0x7f85f0101890 2024-04-25T20:50:37.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85eeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 0x7f85f0101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:37.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85eeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 0x7f85f0101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:36392/0 (socket says 172.21.15.186:36392) 2024-04-25T20:50:37.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85eeffd700 1 -- 172.21.15.186:0/4131889862 learned_addr learned my addr 172.21.15.186:0/4131889862 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:50:37.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85eeffd700 1 -- 172.21.15.186:0/4131889862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 msgr2=0x7f85f0101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T20:50:37.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85eeffd700 1 -- 172.21.15.186:0/4131889862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 msgr2=0x7f85f0101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T20:50:37.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85eeffd700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 0x7f85f0101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T20:50:37.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85ef7fe700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 0x7f85f0101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:37.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85eeffd700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 0x7f85f0101d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:50:37.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85ef7fe700 1 -- 172.21.15.186:0/4131889862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 msgr2=0x7f85f0101d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:50:37.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85ef7fe700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 0x7f85f0101d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:37.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85ef7fe700 1 -- 172.21.15.186:0/4131889862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f0103270 con 0x7f85f01036f0 2024-04-25T20:50:37.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.836+0000 7f85ef7fe700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 0x7f85f0101350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f85e0014000 tx=0x7f85e0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:37.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.837+0000 7f85ecff9700 1 -- 172.21.15.186:0/4131889862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f85e0005dc0 con 0x7f85f01036f0 2024-04-25T20:50:37.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.837+0000 7f85ecff9700 1 -- 172.21.15.186:0/4131889862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e000be10 con 0x7f85f01036f0 2024-04-25T20:50:37.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.837+0000 7f85ecff9700 1 -- 172.21.15.186:0/4131889862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f85e0022e30 con 0x7f85f01036f0 2024-04-25T20:50:37.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.837+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f01a44d0 con 0x7f85f01036f0 2024-04-25T20:50:37.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.837+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f01a49b0 con 0x7f85f01036f0 2024-04-25T20:50:37.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.838+0000 7f85ecff9700 1 -- 172.21.15.186:0/4131889862 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f85e002a070 con 0x7f85f01036f0 2024-04-25T20:50:37.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.838+0000 7f85ecff9700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f85d8039440 0x7f85d803b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:37.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.838+0000 7f85ecff9700 1 -- 172.21.15.186:0/4131889862 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f85e001e030 con 0x7f85f01036f0 2024-04-25T20:50:37.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.838+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f006aca0 con 0x7f85f01036f0 2024-04-25T20:50:37.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.838+0000 7f85eeffd700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f85d8039440 0x7f85d803b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:37.843 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.838+0000 7f85eeffd700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f85d8039440 0x7f85d803b900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009180 tx=0x7f85e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:37.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:37.844+0000 7f85ecff9700 1 -- 172.21.15.186:0/4131889862 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85e0013070 con 0x7f85f01036f0 2024-04-25T20:50:38.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.125+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f85f0102950 con 0x7f85f01036f0 2024-04-25T20:50:38.126 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.125+0000 7f85ecff9700 1 -- 172.21.15.186:0/4131889862 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f85e002b3b0 con 0x7f85f01036f0 2024-04-25T20:50:38.126 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:50:38.126 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","modified":"2024-04-25T20:50:29.613167Z","created":"2024-04-25T20:48:48.454006Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f85d8039440 msgr2=0x7f85d803b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f85d8039440 0x7f85d803b900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009180 tx=0x7f85e4008040 comp rx=0 tx=0).stop 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 msgr2=0x7f85f0101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 0x7f85f0101350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f85e0014000 tx=0x7f85e0005030 comp rx=0 tx=0).stop 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 shutdown_connections 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f85d8039440 0x7f85d803b900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85f01036f0 0x7f85f0101350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 --2- 172.21.15.186:0/4131889862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f0101890 0x7f85f0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:38.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 >> 172.21.15.186:0/4131889862 conn(0x7f85f00fcf20 msgr2=0x7f85f01064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:38.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 shutdown_connections 2024-04-25T20:50:38.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:38.128+0000 7f85f6188700 1 -- 172.21.15.186:0/4131889862 wait complete. 2024-04-25T20:50:38.130 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-04-25T20:50:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 podman[32505]: 2024-04-25 20:50:38.233247155 +0000 UTC m=+0.120056905 container create 68d25731cb93434d4b3ca64f5be5634701472d702d4f54792070758b8b0cfe4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T20:50:38.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 podman[32505]: 2024-04-25 20:50:38.477345876 +0000 UTC m=+0.364155626 container init 68d25731cb93434d4b3ca64f5be5634701472d702d4f54792070758b8b0cfe4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T20:50:38.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 podman[32505]: 2024-04-25 20:50:38.480489967 +0000 UTC m=+0.367299755 container start 68d25731cb93434d4b3ca64f5be5634701472d702d4f54792070758b8b0cfe4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T20:50:38.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T20:50:38.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T20:50:38.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pidfile_write: ignore empty --pid-file 2024-04-25T20:50:38.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: load: jerasure load: lrc 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: RocksDB version: 7.9.2 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Git sha 0 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: DB SUMMARY 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: DB Session ID: 3FCCUJT8VHLLBE5M12K0 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: CURRENT file: CURRENT 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.error_if_exists: 0 2024-04-25T20:50:38.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.create_if_missing: 0 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.paranoid_checks: 1 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.env: 0x562c86343cc0 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.info_log: 0x562c86d83be0 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.statistics: (nil) 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.use_fsync: 0 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_log_file_size: 0 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T20:50:38.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.allow_fallocate: 1 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.use_direct_reads: 0 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.db_log_dir: 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.wal_dir: 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T20:50:38.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T20:50:38.787 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T20:50:38.787 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T20:50:38.787 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T20:50:38.787 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.write_buffer_manager: 0x562c86d170e0 2024-04-25T20:50:38.787 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T20:50:38.789 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T20:50:38.789 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T20:50:38.789 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T20:50:38.789 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T20:50:38.789 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T20:50:38.789 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T20:50:38.789 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T20:50:38.790 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.unordered_write: 0 2024-04-25T20:50:38.790 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T20:50:38.790 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T20:50:38.790 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T20:50:38.790 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T20:50:38.790 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.row_cache: None 2024-04-25T20:50:38.790 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.wal_filter: None 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.two_write_queues: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.wal_compression: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.atomic_flush: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.log_readahead_size: 0 2024-04-25T20:50:38.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_background_jobs: 2 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_background_compactions: -1 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_subcompactions: 1 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T20:50:38.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_open_files: -1 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_background_flushes: -1 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Compression algorithms supported: 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: kZSTD supported: 0 2024-04-25T20:50:38.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: kXpressCompression supported: 0 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: kZlibCompression supported: 1 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: kSnappyCompression supported: 1 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: kLZ4Compression supported: 1 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: kBZip2Compression supported: 0 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.merge_operator: 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_filter: None 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T20:50:38.795 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562c86d83dc0) 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_type: 0 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: data_block_index_type: 0 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_shortening: 1 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: checksum: 4 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: no_block_cache: 0 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache: 0x562c86d491f0 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-04-25T20:50:38.796 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_options: 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: capacity : 536870912 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: num_shard_bits : 4 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: strict_capacity_limit : 0 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_compressed: (nil) 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: persistent_cache: (nil) 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_size: 4096 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_size_deviation: 10 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_restart_interval: 16 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_block_restart_interval: 1 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: metadata_block_size: 4096 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: partition_filters: 0 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: use_delta_encoding: 1 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: filter_policy: bloomfilter 2024-04-25T20:50:38.797 INFO:journalctl@ceph.mon.b.smithi092.stdout: whole_key_filtering: 1 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: verify_compression: 0 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: format_version: 5 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: enable_index_compression: 1 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_align: 0 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: max_auto_readahead_size: 262144 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: prepopulate_block_cache: 0 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: initial_auto_readahead_size: 8192 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression: NoCompression 2024-04-25T20:50:38.798 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.num_levels: 7 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T20:50:38.799 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T20:50:38.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T20:50:38.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T20:50:38.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.inplace_update_support: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.bloom_locality: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.max_successive_merges: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.ttl: 2592000 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T20:50:38.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.enable_blob_files: false 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.min_blob_size: 0 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 67e99a11-a3f4-429f-b0be-8d67e50d707e 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078238510252, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078238510667, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714078238, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "67e99a11-a3f4-429f-b0be-8d67e50d707e", "db_session_id": "3FCCUJT8VHLLBE5M12K0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078238510753, "job": 1, "event": "recovery_finished"} 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562c86e5e000 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: DB pointer 0x562c86e48000 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** DB Stats ** 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:50:38.805 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** Compaction Stats [default] ** 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** Compaction Stats [default] ** 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T20:50:38.806 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: Block cache BinnedLRUCache@0x562c86d491f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: using public_addr v2:172.21.15.92:0/0 -> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] 2024-04-25T20:50:38.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: starting mon.b rank -1 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(???) e0 preinit fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout: e1 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout: legacy client fscid: -1 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout: No filesystems configured 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T20:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Deploying cephadm binary to smithi186 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Added host smithi186 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1890546780' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1890546780' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.812 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c;count:3 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Deploying daemon mon.c on smithi186 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.a calling monitor election 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/1224724809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.c calling monitor election 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:50:38.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: monmap epoch 2 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: last_changed 2024-04-25T20:50:29.613167+0000 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: min_mon_release 19 (squid) 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: election_strategy: 1 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: fsmap 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mgrmap e13: x(active, since 48s) 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: overall HEALTH_OK 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:38.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:38.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: Deploying daemon mon.b on smithi092 2024-04-25T20:50:38.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:38.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:38.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 ceph-mon[32524]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T20:50:38.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 bash[32505]: 68d25731cb93434d4b3ca64f5be5634701472d702d4f54792070758b8b0cfe4c 2024-04-25T20:50:38.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:38 smithi092 systemd[1]: Started Ceph mon.b for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:50:39.697 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:50:39.698 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:50:39.941 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:50:43.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:43.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: mon.a calling monitor election 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: mon.c calling monitor election 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:43.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: monmap epoch 3 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: last_changed 2024-04-25T20:50:38.533467+0000 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: election_strategy: 1 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:43.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: fsmap 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: mgrmap e13: x(active, since 57s) 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:43.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: mon.a calling monitor election 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: mon.c calling monitor election 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: monmap epoch 3 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: last_changed 2024-04-25T20:50:38.533467+0000 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: min_mon_release 19 (squid) 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: election_strategy: 1 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: fsmap 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: mgrmap e13: x(active, since 57s) 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: overall HEALTH_OK 2024-04-25T20:50:43.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:43.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:43.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:43.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:44.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.377+0000 7f707cc49700 1 -- 172.21.15.186:0/1847087902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 msgr2=0x7f7060005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:44.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.377+0000 7f707cc49700 1 --2- 172.21.15.186:0/1847087902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 0x7f7060005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7078078b70 tx=0x7f706c00feb0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.378+0000 7f707cc49700 1 -- 172.21.15.186:0/1847087902 shutdown_connections 2024-04-25T20:50:44.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.378+0000 7f707cc49700 1 --2- 172.21.15.186:0/1847087902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 0x7f7060005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.378+0000 7f707cc49700 1 -- 172.21.15.186:0/1847087902 >> 172.21.15.186:0/1847087902 conn(0x7f7078077860 msgr2=0x7f7078077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:44.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.378+0000 7f707cc49700 1 -- 172.21.15.186:0/1847087902 shutdown_connections 2024-04-25T20:50:44.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.378+0000 7f707cc49700 1 -- 172.21.15.186:0/1847087902 wait complete. 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707cc49700 1 Processor -- start 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707cc49700 1 -- start start 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707cc49700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 0x7f7078101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707cc49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7078102370 0x7f70781027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707cc49700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 0x7f70781a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707cc49700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f707810a4c0 con 0x7f70600035c0 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707cc49700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f707810a340 con 0x7f7078102370 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707cc49700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f707810a640 con 0x7f70781a4330 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f7076d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 0x7f70781a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f7076d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 0x7f70781a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:44420/0 (socket says 172.21.15.186:44420) 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f7076d9d700 1 -- 172.21.15.186:0/51584081 learned_addr learned my addr 172.21.15.186:0/51584081 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:50:44.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f7076d9d700 1 -- 172.21.15.186:0/51584081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 msgr2=0x7f70781a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f7076d9d700 1 -- 172.21.15.186:0/51584081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 msgr2=0x7f70781a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f7076d9d700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 0x7f70781a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f7076d9d700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 0x7f70781a6720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.379+0000 7f707659c700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 0x7f7078101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f707659c700 1 -- 172.21.15.186:0/51584081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 msgr2=0x7f70781a6720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f707659c700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 0x7f70781a6720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f707659c700 1 -- 172.21.15.186:0/51584081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7078102370 msgr2=0x7f70781027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f707659c700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7078102370 0x7f70781027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f707659c700 1 -- 172.21.15.186:0/51584081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70781a6e50 con 0x7f70600035c0 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f707659c700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 0x7f7078101e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7078078b70 tx=0x7f706c013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f70677fe700 1 -- 172.21.15.186:0/51584081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c005020 con 0x7f70600035c0 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70781a70e0 con 0x7f70600035c0 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f70677fe700 1 -- 172.21.15.186:0/51584081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f706c0051c0 con 0x7f70600035c0 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f70677fe700 1 -- 172.21.15.186:0/51584081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c003c00 con 0x7f70600035c0 2024-04-25T20:50:44.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.380+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70781a7620 con 0x7f70600035c0 2024-04-25T20:50:44.384 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.381+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70780640a0 con 0x7f70600035c0 2024-04-25T20:50:44.384 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.381+0000 7f70677fe700 1 -- 172.21.15.186:0/51584081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f706c023070 con 0x7f70600035c0 2024-04-25T20:50:44.384 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.381+0000 7f70677fe700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7058039ae0 0x7f705803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:44.384 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.381+0000 7f70677fe700 1 -- 172.21.15.186:0/51584081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f706c04ecc0 con 0x7f70600035c0 2024-04-25T20:50:44.385 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.382+0000 7f7075d9b700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7058039ae0 0x7f705803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:44.386 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.385+0000 7f7075d9b700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7058039ae0 0x7f705803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f70680098e0 tx=0x7f7068006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:44.388 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.387+0000 7f70677fe700 1 -- 172.21.15.186:0/51584081 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f706c028020 con 0x7f70600035c0 2024-04-25T20:50:44.675 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.673+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f70781a78e0 con 0x7f70600035c0 2024-04-25T20:50:44.675 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:44 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:44.675 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:44 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:44.675 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.674+0000 7f70677fe700 1 -- 172.21.15.186:0/51584081 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f706c01ba40 con 0x7f70600035c0 2024-04-25T20:50:44.676 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T20:50:44.676 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","modified":"2024-04-25T20:50:38.533467Z","created":"2024-04-25T20:48:48.454006Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T20:50:44.678 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7058039ae0 msgr2=0x7f705803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:44.678 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7058039ae0 0x7f705803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f70680098e0 tx=0x7f7068006dd0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.678 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 msgr2=0x7f7078101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:44.679 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 0x7f7078101e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7078078b70 tx=0x7f706c013f60 comp rx=0 tx=0).stop 2024-04-25T20:50:44.679 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 shutdown_connections 2024-04-25T20:50:44.679 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7058039ae0 0x7f705803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.679 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70600035c0 0x7f7078101e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.679 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70781a4330 0x7f70781a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.679 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 --2- 172.21.15.186:0/51584081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7078102370 0x7f70781027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:44.679 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.677+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 >> 172.21.15.186:0/51584081 conn(0x7f7078077860 msgr2=0x7f70780ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:44.679 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.678+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 shutdown_connections 2024-04-25T20:50:44.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:44.678+0000 7f707cc49700 1 -- 172.21.15.186:0/51584081 wait complete. 2024-04-25T20:50:44.680 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2024-04-25T20:50:44.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:44 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:44.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:44 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:45.696 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T20:50:45.696 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T20:50:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: mon.a calling monitor election 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: mon.c calling monitor election 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:46.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: monmap epoch 3 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: last_changed 2024-04-25T20:50:38.533467+0000 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: min_mon_release 19 (squid) 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: election_strategy: 1 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-04-25T20:50:46.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: fsmap 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: mgrmap e13: x(active, since 57s) 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: overall HEALTH_OK 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:46.035 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:46.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: mon.b calling monitor election 2024-04-25T20:50:46.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-25T20:50:46.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T20:50:46.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T20:50:46.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: mon.b calling monitor election 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: mon.c calling monitor election 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: mon.a calling monitor election 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: monmap epoch 3 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: last_changed 2024-04-25T20:50:38.533467+0000 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: election_strategy: 1 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-04-25T20:50:46.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: fsmap 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: mgrmap e13: x(active, since 59s) 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:46.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:46.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:46.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: mon.b calling monitor election 2024-04-25T20:50:46.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-25T20:50:46.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T20:50:46.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T20:50:46.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: mon.b calling monitor election 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: mon.c calling monitor election 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: mon.a calling monitor election 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: monmap epoch 3 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: last_changed 2024-04-25T20:50:38.533467+0000 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: min_mon_release 19 (squid) 2024-04-25T20:50:46.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: election_strategy: 1 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: fsmap 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: mgrmap e13: x(active, since 59s) 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: overall HEALTH_OK 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:46.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:46.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: mon.b calling monitor election 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: mon.b calling monitor election 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: mon.c calling monitor election 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: mon.a calling monitor election 2024-04-25T20:50:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: monmap epoch 3 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: last_changed 2024-04-25T20:50:38.533467+0000 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: created 2024-04-25T20:48:48.454006+0000 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: min_mon_release 19 (squid) 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: election_strategy: 1 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: fsmap 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: mgrmap e13: x(active, since 59s) 2024-04-25T20:50:46.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: overall HEALTH_OK 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:46.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:47 smithi047 ceph-mon[29195]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:47.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:47 smithi047 ceph-mon[29195]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:50:47.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:47 smithi047 ceph-mon[29195]: Reconfiguring daemon mon.a on smithi047 2024-04-25T20:50:47.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:47.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:47 smithi186 ceph-mon[32529]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:47.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:47 smithi186 ceph-mon[32529]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:50:47.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:47 smithi186 ceph-mon[32529]: Reconfiguring daemon mon.a on smithi047 2024-04-25T20:50:47.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:47.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:47 smithi092 ceph-mon[32524]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:47.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:47 smithi092 ceph-mon[32524]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:50:47.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:47 smithi092 ceph-mon[32524]: Reconfiguring daemon mon.a on smithi047 2024-04-25T20:50:47.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:50:47.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:50:47 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:50:47.533+0000 7f98465d7700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T20:50:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 -- 172.21.15.47:0/4032074622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd20074e10 msgr2=0x7efd200751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 --2- 172.21.15.47:0/4032074622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd20074e10 0x7efd200751f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd10009910 tx=0x7efd10009c20 comp rx=0 tx=0).stop 2024-04-25T20:50:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 -- 172.21.15.47:0/4032074622 shutdown_connections 2024-04-25T20:50:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 --2- 172.21.15.47:0/4032074622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd20074e10 0x7efd200751f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 -- 172.21.15.47:0/4032074622 >> 172.21.15.47:0/4032074622 conn(0x7efd200fdf80 msgr2=0x7efd201003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 -- 172.21.15.47:0/4032074622 shutdown_connections 2024-04-25T20:50:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 -- 172.21.15.47:0/4032074622 wait complete. 2024-04-25T20:50:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 Processor -- start 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.423+0000 7efd26854700 1 -- start start 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd26854700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd20074e10 0x7efd2019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd26854700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2019a950 0x7efd2019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd26854700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd2019f180 0x7efd2019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd26854700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010eba0 con 0x7efd2019f180 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd26854700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010ea20 con 0x7efd2019a950 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd26854700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010ed20 con 0x7efd20074e10 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd24df1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd2019f180 0x7efd2019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd24df1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd2019f180 0x7efd2019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34400/0 (socket says 172.21.15.47:34400) 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd24df1700 1 -- 172.21.15.47:0/1200589969 learned_addr learned my addr 172.21.15.47:0/1200589969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd1f7fe700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2019a950 0x7efd2019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd1ffff700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd20074e10 0x7efd2019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd24df1700 1 -- 172.21.15.47:0/1200589969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd20074e10 msgr2=0x7efd2019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd24df1700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd20074e10 0x7efd2019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd24df1700 1 -- 172.21.15.47:0/1200589969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2019a950 msgr2=0x7efd2019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd24df1700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2019a950 0x7efd2019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.424+0000 7efd24df1700 1 -- 172.21.15.47:0/1200589969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd201a6ef0 con 0x7efd2019f180 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.425+0000 7efd24df1700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd2019f180 0x7efd2019f630 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efd1800ba50 tx=0x7efd1800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.425+0000 7efd1d7fa700 1 -- 172.21.15.47:0/1200589969 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18019070 con 0x7efd2019f180 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.425+0000 7efd1d7fa700 1 -- 172.21.15.47:0/1200589969 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd18014440 con 0x7efd2019f180 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.425+0000 7efd1d7fa700 1 -- 172.21.15.47:0/1200589969 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1800ce60 con 0x7efd2019f180 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.425+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd201a7120 con 0x7efd2019f180 2024-04-25T20:50:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.425+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd201a7610 con 0x7efd2019f180 2024-04-25T20:50:48.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.426+0000 7efd1d7fa700 1 -- 172.21.15.47:0/1200589969 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efd1801c070 con 0x7efd2019f180 2024-04-25T20:50:48.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.426+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd200640a0 con 0x7efd2019f180 2024-04-25T20:50:48.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.426+0000 7efd1d7fa700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7efd08039ae0 0x7efd0803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:48.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.426+0000 7efd1d7fa700 1 -- 172.21.15.47:0/1200589969 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efd1804bdd0 con 0x7efd2019f180 2024-04-25T20:50:48.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.426+0000 7efd1ffff700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7efd08039ae0 0x7efd0803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:48.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.427+0000 7efd1ffff700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7efd08039ae0 0x7efd0803bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd10009750 tx=0x7efd1001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:48.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.431+0000 7efd1d7fa700 1 -- 172.21.15.47:0/1200589969 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd1804f030 con 0x7efd2019f180 2024-04-25T20:50:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.637+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efd201a7900 con 0x7efd2019f180 2024-04-25T20:50:48.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.637+0000 7efd1d7fa700 1 -- 172.21.15.47:0/1200589969 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7efd18014910 con 0x7efd2019f180 2024-04-25T20:50:48.638 INFO:teuthology.orchestra.run.smithi047.stdout:# minimal ceph.conf for 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:48.638 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-04-25T20:50:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:50:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: mon_host = [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7efd08039ae0 msgr2=0x7efd0803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7efd08039ae0 0x7efd0803bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd10009750 tx=0x7efd1001a040 comp rx=0 tx=0).stop 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd2019f180 msgr2=0x7efd2019f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd2019f180 0x7efd2019f630 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efd1800ba50 tx=0x7efd1800bd60 comp rx=0 tx=0).stop 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 shutdown_connections 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7efd08039ae0 0x7efd0803bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd2019f180 0x7efd2019f630 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd20074e10 0x7efd2019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 --2- 172.21.15.47:0/1200589969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2019a950 0x7efd2019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 >> 172.21.15.47:0/1200589969 conn(0x7efd200fdf80 msgr2=0x7efd20108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 shutdown_connections 2024-04-25T20:50:48.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:48.640+0000 7efd26854700 1 -- 172.21.15.47:0/1200589969 wait complete. 2024-04-25T20:50:49.280 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T20:50:49.280 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:50:49.280 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:50:49.309 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:50:49.309 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:49.374 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:50:49.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: Reconfiguring daemon mon.b on smithi092 2024-04-25T20:50:49.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:49 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1200589969' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:49.401 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:50:49.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:49.464 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:50:49.464 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:50:49.495 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:50:49.496 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:50:49.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:49.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:49.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:50:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: Reconfiguring daemon mon.b on smithi092 2024-04-25T20:50:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:49 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1200589969' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: Reconfiguring daemon mon.b on smithi092 2024-04-25T20:50:49.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:49 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1200589969' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:49.564 INFO:tasks.cephadm:Adding mgr.x on smithi047 2024-04-25T20:50:49.564 INFO:tasks.cephadm:Adding mgr.y on smithi092 2024-04-25T20:50:49.564 INFO:tasks.cephadm:Adding mgr.z on smithi186 2024-04-25T20:50:49.564 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi047=x;smithi092=y;smithi186=z' 2024-04-25T20:50:49.869 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:50:50.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:50 smithi186 ceph-mon[32529]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:50.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:50.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:50.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:50 smithi186 ceph-mon[32529]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:50:50.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:50.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:50.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:50.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:50 smithi186 ceph-mon[32529]: Reconfiguring daemon mon.c on smithi186 2024-04-25T20:50:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:50 smithi047 ceph-mon[29195]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:50 smithi047 ceph-mon[29195]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:50:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:51.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:51.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:51.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:50 smithi047 ceph-mon[29195]: Reconfiguring daemon mon.c on smithi186 2024-04-25T20:50:51.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:50 smithi092 ceph-mon[32524]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:51.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:51.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:51.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:50 smithi092 ceph-mon[32524]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:50:51.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:50:51.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:50:51.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:51.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:50 smithi092 ceph-mon[32524]: Reconfiguring daemon mon.c on smithi186 2024-04-25T20:50:52.253 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 -- 172.21.15.186:0/3574192436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc102ba0 msgr2=0x7f4ffc103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:52.253 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 --2- 172.21.15.186:0/3574192436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc102ba0 0x7f4ffc103000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009990 tx=0x7f4fe8009ca0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.254 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 -- 172.21.15.186:0/3574192436 shutdown_connections 2024-04-25T20:50:52.254 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 --2- 172.21.15.186:0/3574192436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ffc103540 0x7f4ffc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.254 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 --2- 172.21.15.186:0/3574192436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ffc1019a0 0x7f4ffc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.254 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 --2- 172.21.15.186:0/3574192436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc102ba0 0x7f4ffc103000 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.254 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 -- 172.21.15.186:0/3574192436 >> 172.21.15.186:0/3574192436 conn(0x7f4ffc0fcf60 msgr2=0x7f4ffc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:52.254 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 -- 172.21.15.186:0/3574192436 shutdown_connections 2024-04-25T20:50:52.254 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.252+0000 7f5002b0f700 1 -- 172.21.15.186:0/3574192436 wait complete. 2024-04-25T20:50:52.256 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f5002b0f700 1 Processor -- start 2024-04-25T20:50:52.256 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f5002b0f700 1 -- start start 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f5002b0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ffc1019a0 0x7f4ffc06bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f5002b0f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ffc102ba0 0x7f4ffc19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f5002b0f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc103540 0x7f4ffc19ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f5002b0f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc10d410 con 0x7f4ffc102ba0 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f5002b0f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc10d290 con 0x7f4ffc1019a0 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f5002b0f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc10d590 con 0x7f4ffc103540 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f50010ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc103540 0x7f4ffc19ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f50010ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc103540 0x7f4ffc19ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:44438/0 (socket says 172.21.15.186:44438) 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f50010ac700 1 -- 172.21.15.186:0/2412841356 learned_addr learned my addr 172.21.15.186:0/2412841356 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.253+0000 7f4ffbfff700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ffc102ba0 0x7f4ffc19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f50008ab700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ffc1019a0 0x7f4ffc06bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f50010ac700 1 -- 172.21.15.186:0/2412841356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ffc1019a0 msgr2=0x7f4ffc06bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:52.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f50010ac700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ffc1019a0 0x7f4ffc06bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f50010ac700 1 -- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ffc102ba0 msgr2=0x7f4ffc19d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f50010ac700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ffc102ba0 0x7f4ffc19d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f50010ac700 1 -- 172.21.15.186:0/2412841356 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc1a07e0 con 0x7f4ffc103540 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f50008ab700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ffc1019a0 0x7f4ffc06bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f50010ac700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc103540 0x7f4ffc19ff10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400ef30 tx=0x7f4ff400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f4ff9ffb700 1 -- 172.21.15.186:0/2412841356 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40092e0 con 0x7f4ffc103540 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f4ff9ffb700 1 -- 172.21.15.186:0/2412841356 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff4004bb0 con 0x7f4ffc103540 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ffc1a0a70 con 0x7f4ffc103540 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f4ff9ffb700 1 -- 172.21.15.186:0/2412841356 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4004280 con 0x7f4ffc103540 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.254+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ffc1a0fb0 con 0x7f4ffc103540 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.255+0000 7f4ff9ffb700 1 -- 172.21.15.186:0/2412841356 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4ff401b030 con 0x7f4ffc103540 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.255+0000 7f4ff9ffb700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4fec039890 0x7f4fec03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:52.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.255+0000 7f4ff9ffb700 1 -- 172.21.15.186:0/2412841356 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4ff404c5a0 con 0x7f4ffc103540 2024-04-25T20:50:52.259 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.255+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ffc069730 con 0x7f4ffc103540 2024-04-25T20:50:52.259 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.255+0000 7f50008ab700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4fec039890 0x7f4fec03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:52.259 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.256+0000 7f50008ab700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4fec039890 0x7f4fec03bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005fd0 tx=0x7f4ff0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:52.261 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.261+0000 7f4ff9ffb700 1 -- 172.21.15.186:0/2412841356 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ff4022020 con 0x7f4ffc103540 2024-04-25T20:50:52.477 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.476+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi092=y;smithi186=z", "target": ["mon-mgr", ""]}) -- 0x7f4ffc062f10 con 0x7f4fec039890 2024-04-25T20:50:52.482 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.481+0000 7f4ff9ffb700 1 -- 172.21.15.186:0/2412841356 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4ffc062f10 con 0x7f4fec039890 2024-04-25T20:50:52.482 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2024-04-25T20:50:52.484 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4fec039890 msgr2=0x7f4fec03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:52.484 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4fec039890 0x7f4fec03bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005fd0 tx=0x7f4ff0005dc0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.484 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc103540 msgr2=0x7f4ffc19ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:52.484 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc103540 0x7f4ffc19ff10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400ef30 tx=0x7f4ff400c3a0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.484 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 shutdown_connections 2024-04-25T20:50:52.484 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4fec039890 0x7f4fec03bd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.484 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc103540 0x7f4ffc19ff10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.485 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ffc1019a0 0x7f4ffc06bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.485 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 --2- 172.21.15.186:0/2412841356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ffc102ba0 0x7f4ffc19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.485 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 >> 172.21.15.186:0/2412841356 conn(0x7f4ffc0fcf60 msgr2=0x7f4ffc105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:52.485 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 shutdown_connections 2024-04-25T20:50:52.485 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:50:52.483+0000 7f5002b0f700 1 -- 172.21.15.186:0/2412841356 wait complete. 2024-04-25T20:50:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:52.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:52.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:52.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:52.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:52.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:52.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:52.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:52.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:52.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:52.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:53.038 DEBUG:teuthology.orchestra.run.smithi092:mgr.y> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T20:50:53.041 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T20:50:53.043 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T20:50:53.044 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:50:53.044 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:50:53.063 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:50:53.064 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-04-25T20:50:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:50:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:53.123 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T20:50:53.123 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:53.123 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:53.123 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-25 20:49:55.999576621 +0000 2024-04-25T20:50:53.123 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-25 20:45:20.007237109 +0000 2024-04-25T20:50:53.124 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-25 20:45:20.007237109 +0000 2024-04-25T20:50:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-25 20:45:20.007237109 +0000 2024-04-25T20:50:53.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:50:53.193 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-25T20:50:53.193 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-25T20:50:53.193 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000131558 s, 3.9 MB/s 2024-04-25T20:50:53.195 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:50:53.253 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-04-25T20:50:53.311 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:50:53.312 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:53.312 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T20:50:53.312 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:53.312 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:53.312 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-25 20:49:56.027576039 +0000 2024-04-25T20:50:53.312 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-25 20:45:20.351229718 +0000 2024-04-25T20:50:53.312 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-25 20:45:20.351229718 +0000 2024-04-25T20:50:53.312 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-25 20:45:20.351229718 +0000 2024-04-25T20:50:53.312 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:50:53.378 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-25T20:50:53.379 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-25T20:50:53.379 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000133166 s, 3.8 MB/s 2024-04-25T20:50:53.380 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:50:53.439 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-04-25T20:50:53.497 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:50:53.497 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:53.497 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T20:50:53.497 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:53.498 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:53.498 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-25 20:49:56.053575498 +0000 2024-04-25T20:50:53.498 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-25 20:45:20.682222606 +0000 2024-04-25T20:50:53.498 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-25 20:45:20.682222606 +0000 2024-04-25T20:50:53.498 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-25 20:45:20.682222606 +0000 2024-04-25T20:50:53.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:50:53.563 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-25T20:50:53.563 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-25T20:50:53.563 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000117918 s, 4.3 MB/s 2024-04-25T20:50:53.565 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:50:53.622 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-25 20:49:56.080574937 +0000 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-25 20:45:21.010215558 +0000 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-25 20:45:21.010215558 +0000 2024-04-25T20:50:53.680 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-25 20:45:21.010215558 +0000 2024-04-25T20:50:53.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi092=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: Saving service mgr spec with placement smithi047=x;smithi092=y;smithi186=z;count:3 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:50:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:53.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:53 smithi186 ceph-mon[32529]: Deploying daemon mgr.z on smithi186 2024-04-25T20:50:53.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:53.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi092=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:53.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: Saving service mgr spec with placement smithi047=x;smithi092=y;smithi186=z;count:3 2024-04-25T20:50:53.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:53.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:53 smithi047 ceph-mon[29195]: Deploying daemon mgr.z on smithi186 2024-04-25T20:50:53.744 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-25T20:50:53.744 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-25T20:50:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000100928 s, 5.1 MB/s 2024-04-25T20:50:53.746 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:50:53.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:53.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi092=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:53.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: Saving service mgr spec with placement smithi047=x;smithi092=y;smithi186=z;count:3 2024-04-25T20:50:53.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:53.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:53.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:53.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:53.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:53.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:50:53.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:50:53.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:50:53.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:53.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:53 smithi092 ceph-mon[32524]: Deploying daemon mgr.z on smithi186 2024-04-25T20:50:53.804 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-25T20:50:53.804 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:50:53.822 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:50:53.822 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-04-25T20:50:53.879 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:50:53.879 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T20:50:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-25 20:50:37.913209888 +0000 2024-04-25T20:50:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-25 20:45:33.765085480 +0000 2024-04-25T20:50:53.880 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-25 20:45:33.765085480 +0000 2024-04-25T20:50:53.880 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-25 20:45:33.765085480 +0000 2024-04-25T20:50:53.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:50:53.944 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-25T20:50:53.944 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-25T20:50:53.944 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000261181 s, 2.0 MB/s 2024-04-25T20:50:53.946 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:50:54.003 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-25 20:50:37.913209888 +0000 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-25 20:45:34.087076380 +0000 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-25 20:45:34.087076380 +0000 2024-04-25T20:50:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-25 20:45:34.087076380 +0000 2024-04-25T20:50:54.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:50:54.123 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-25T20:50:54.123 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-25T20:50:54.123 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00021857 s, 2.3 MB/s 2024-04-25T20:50:54.124 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:50:54.181 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-04-25T20:50:54.237 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:50:54.237 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:54.238 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T20:50:54.238 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:54.238 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:54.238 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-25 20:50:37.914209861 +0000 2024-04-25T20:50:54.238 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-25 20:45:34.432066630 +0000 2024-04-25T20:50:54.238 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-25 20:45:34.432066630 +0000 2024-04-25T20:50:54.238 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-25 20:45:34.432066630 +0000 2024-04-25T20:50:54.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:50:54.300 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-25T20:50:54.300 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-25T20:50:54.300 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000217966 s, 2.3 MB/s 2024-04-25T20:50:54.301 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:50:54.358 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-25 20:50:37.914209861 +0000 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-25 20:45:34.766057191 +0000 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-25 20:45:34.766057191 +0000 2024-04-25T20:50:54.413 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-25 20:45:34.766057191 +0000 2024-04-25T20:50:54.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:50:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-25T20:50:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-25T20:50:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000198884 s, 2.6 MB/s 2024-04-25T20:50:54.477 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:50:54.534 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T20:50:54.534 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:50:54.550 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:50:54.551 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-04-25T20:50:54.606 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:50:54.606 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:54.606 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T20:50:54.606 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:54.607 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:54.607 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-25 20:50:54.416182424 +0000 2024-04-25T20:50:54.607 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-25 20:45:33.438221300 +0000 2024-04-25T20:50:54.607 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-25 20:45:33.438221300 +0000 2024-04-25T20:50:54.607 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-25 20:45:33.438221300 +0000 2024-04-25T20:50:54.607 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:50:54.668 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:54 smithi186 systemd[1]: Starting Ceph mgr.z for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:50:54.672 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-25T20:50:54.672 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-25T20:50:54.672 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000266589 s, 1.9 MB/s 2024-04-25T20:50:54.673 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:50:54.729 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-04-25T20:50:54.784 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:50:54.785 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:54.785 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T20:50:54.785 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:54.785 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:54.785 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-25 20:50:54.416182424 +0000 2024-04-25T20:50:54.785 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-25 20:45:33.771214720 +0000 2024-04-25T20:50:54.785 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-25 20:45:33.771214720 +0000 2024-04-25T20:50:54.785 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-25 20:45:33.771214720 +0000 2024-04-25T20:50:54.785 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:50:54.853 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-25T20:50:54.853 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-25T20:50:54.854 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.00613904 s, 83.4 kB/s 2024-04-25T20:50:54.855 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:50:54.908 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-04-25T20:50:54.941 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:54 smithi186 podman[34199]: 2024-04-25 20:50:54.595948176 +0000 UTC m=+0.017798666 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:50:54.941 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:54 smithi186 podman[34199]: 2024-04-25 20:50:54.730095868 +0000 UTC m=+0.151946384 container create 0bac3ee556bb1ca41b47ac20d63a718c48192a26dd4f5284a17bc2b3795a9bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T20:50:54.941 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:54 smithi186 podman[34199]: 2024-04-25 20:50:54.941017786 +0000 UTC m=+0.362868262 container init 0bac3ee556bb1ca41b47ac20d63a718c48192a26dd4f5284a17bc2b3795a9bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-25 20:50:54.416182424 +0000 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-25 20:45:34.119207843 +0000 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-25 20:45:34.119207843 +0000 2024-04-25T20:50:54.955 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-25 20:45:34.119207843 +0000 2024-04-25T20:50:54.956 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:50:55.018 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-25T20:50:55.018 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-25T20:50:55.018 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000226353 s, 2.3 MB/s 2024-04-25T20:50:55.019 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:50:55.075 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-04-25T20:50:55.130 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:50:55.130 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:50:55.131 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T20:50:55.131 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:50:55.131 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:50:55.131 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-25 20:50:54.417182407 +0000 2024-04-25T20:50:55.131 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-25 20:45:34.469200926 +0000 2024-04-25T20:50:55.131 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-25 20:45:34.469200926 +0000 2024-04-25T20:50:55.131 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-25 20:45:34.469200926 +0000 2024-04-25T20:50:55.131 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:50:55.191 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:54 smithi186 podman[34199]: 2024-04-25 20:50:54.943939917 +0000 UTC m=+0.365790399 container start 0bac3ee556bb1ca41b47ac20d63a718c48192a26dd4f5284a17bc2b3795a9bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T20:50:55.191 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:54 smithi186 ceph-mgr[34325]: -- 172.21.15.186:0/234263598 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55807124a5a0 con 0x55807121f000 2024-04-25T20:50:55.191 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:55 smithi186 bash[34199]: 0bac3ee556bb1ca41b47ac20d63a718c48192a26dd4f5284a17bc2b3795a9bb6 2024-04-25T20:50:55.191 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:55 smithi186 systemd[1]: Started Ceph mgr.z for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:50:55.192 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:55.098+0000 7effe8ec0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:50:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-25T20:50:55.194 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-25T20:50:55.194 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.00020332 s, 2.5 MB/s 2024-04-25T20:50:55.195 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:50:55.250 INFO:tasks.cephadm:Deploying osd.0 on smithi047 with /dev/vg_nvme/lv_4... 2024-04-25T20:50:55.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:50:55.470 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:55.291+0000 7effe8ec0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:50:55.471 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:55.391+0000 7effe8ec0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:50:55.502 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:55 smithi047 ceph-mon[29195]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:50:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:50:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:50:55.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:55.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:55 smithi092 ceph-mon[32524]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:55.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:50:55.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:50:55.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:50:55.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:55.790 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:55.701+0000 7effe8ec0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:50:55.790 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:55.787+0000 7effe8ec0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:50:55.790 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-mon[32529]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:55.790 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.790 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.790 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:55.791 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:50:55.791 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:50:55.791 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:50:55.791 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:56.523 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:56 smithi186 ceph-mon[32529]: Deploying daemon mgr.y on smithi092 2024-04-25T20:50:56.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:56 smithi047 ceph-mon[29195]: Deploying daemon mgr.y on smithi092 2024-04-25T20:50:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:56 smithi092 ceph-mon[32524]: Deploying daemon mgr.y on smithi092 2024-04-25T20:50:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:50:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:50:57.350 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-04-25T20:50:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-04-25T20:50:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00883175 s, 1.2 GB/s 2024-04-25T20:50:57.351 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 2024-04-25T20:50:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-04-25T20:50:57.375 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:57.025+0000 7effe8ec0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:50:57.375 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:57.198+0000 7effe8ec0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:50:57.375 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:57.374+0000 7effe8ec0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:50:57.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:57 smithi047 ceph-mon[29195]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:57.667 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:57 smithi186 ceph-mon[32529]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:57.770 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:57 smithi092 ceph-mon[32524]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:57.771 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:50:57 smithi092 systemd[1]: Started Ceph mgr.y for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:50:57.968 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:57.665+0000 7effe8ec0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:50:57.968 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:57.758+0000 7effe8ec0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:50:57.968 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:57.967+0000 7effe8ec0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:50:58.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:50:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:50:57.769+0000 7f49b25c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:50:58.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:50:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:50:57.957+0000 7f49b25c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:50:58.081 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-04-25T20:50:58.221 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:58.167+0000 7effe8ec0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:50:58.322 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:58.390 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:50:58.065+0000 7f49b25c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:50:58.390 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:50:58.389+0000 7f49b25c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:50:58.711 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:50:58.487+0000 7f49b25c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:50:58.970 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:58.822+0000 7effe8ec0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:50:58.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:58.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:58.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:58.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:58.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:58 smithi186 ceph-mon[32529]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:58.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:50:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:59.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:59.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:59.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:59.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:59.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-mon[32524]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:59.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:50:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:59.092 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:59.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:59.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:59.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:50:59.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:58 smithi047 ceph-mon[29195]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:50:59.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:50:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:59.561 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:59 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:59.260+0000 7effe8ec0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:50:59.561 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:59 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:59.358+0000 7effe8ec0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:50:59.561 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:50:59 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:50:59.560+0000 7effe8ec0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.927+0000 7f3ca8048700 1 -- 172.21.15.47:0/2966812921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca0103550 msgr2=0x7f3ca0105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.927+0000 7f3ca8048700 1 --2- 172.21.15.47:0/2966812921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca0103550 0x7f3ca0105940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009990 tx=0x7f3c9c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.927+0000 7f3ca8048700 1 -- 172.21.15.47:0/2966812921 shutdown_connections 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.927+0000 7f3ca8048700 1 --2- 172.21.15.47:0/2966812921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca0103550 0x7f3ca0105940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.927+0000 7f3ca8048700 1 --2- 172.21.15.47:0/2966812921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca01087b0 0x7f3ca010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.927+0000 7f3ca8048700 1 --2- 172.21.15.47:0/2966812921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca0105e80 0x7f3ca0108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.927+0000 7f3ca8048700 1 -- 172.21.15.47:0/2966812921 >> 172.21.15.47:0/2966812921 conn(0x7f3ca00fcf60 msgr2=0x7f3ca00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.928+0000 7f3ca8048700 1 -- 172.21.15.47:0/2966812921 shutdown_connections 2024-04-25T20:50:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.928+0000 7f3ca8048700 1 -- 172.21.15.47:0/2966812921 wait complete. 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.928+0000 7f3ca8048700 1 Processor -- start 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.928+0000 7f3ca8048700 1 -- start start 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca8048700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca0103550 0x7f3ca019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca8048700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca0105e80 0x7f3ca019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca8048700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca01087b0 0x7f3ca01a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca8048700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010e560 con 0x7f3ca01087b0 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca8048700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010e3e0 con 0x7f3ca0105e80 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca8048700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010e6e0 con 0x7f3ca0103550 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca5de4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca0103550 0x7f3ca019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca65e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca01087b0 0x7f3ca01a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca5de4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca0103550 0x7f3ca019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:48726/0 (socket says 172.21.15.47:48726) 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca55e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca0105e80 0x7f3ca019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca5de4700 1 -- 172.21.15.47:0/3239563782 learned_addr learned my addr 172.21.15.47:0/3239563782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:50:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca65e5700 1 -- 172.21.15.47:0/3239563782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca0103550 msgr2=0x7f3ca019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca65e5700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca0103550 0x7f3ca019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca65e5700 1 -- 172.21.15.47:0/3239563782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca0105e80 msgr2=0x7f3ca019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca65e5700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca0105e80 0x7f3ca019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca65e5700 1 -- 172.21.15.47:0/3239563782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca01a2a40 con 0x7f3ca01087b0 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca5de4700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca0103550 0x7f3ca019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca65e5700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca01087b0 0x7f3ca01a23a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c98007f10 tx=0x7f3c9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98004db0 con 0x7f3ca01087b0 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c98009e20 con 0x7f3ca01087b0 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.929+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca01a2d30 con 0x7f3ca01087b0 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.930+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98015400 con 0x7f3ca01087b0 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.930+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca01a31c0 con 0x7f3ca01087b0 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.930+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3c980155a0 con 0x7f3ca01087b0 2024-04-25T20:50:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.930+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca0069730 con 0x7f3ca01087b0 2024-04-25T20:50:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.931+0000 7f3c96ffd700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3c8c039890 0x7f3c8c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.931+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3c9804c3e0 con 0x7f3ca01087b0 2024-04-25T20:50:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.931+0000 7f3ca5de4700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3c8c039890 0x7f3c8c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.931+0000 7f3ca5de4700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3c8c039890 0x7f3c8c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b5c0 tx=0x7f3c9c003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:50:59.936+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c98010070 con 0x7f3ca01087b0 2024-04-25T20:51:00.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:50:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:50:59.710+0000 7f49b25c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:51:00.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:50:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:50:59.875+0000 7f49b25c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:51:00.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:00.172+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3ca0062e80 con 0x7f3c8c039890 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:51:00.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:51:00.240 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:51:00.240 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:00.240 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:00 smithi047 ceph-mon[29195]: Reconfiguring daemon mgr.x on smithi047 2024-04-25T20:51:00.328 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:00.043+0000 7f49b25c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:51:00.328 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.328 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.328 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:00.329 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-mon[32524]: Reconfiguring daemon mgr.x on smithi047 2024-04-25T20:51:00.417 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:00.084+0000 7effe8ec0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:51:00.417 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:00.176+0000 7effe8ec0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:00.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-mon[32529]: Reconfiguring daemon mgr.x on smithi047 2024-04-25T20:51:00.617 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:00.327+0000 7f49b25c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:51:00.617 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:00.417+0000 7f49b25c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:51:00.617 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:00.616+0000 7f49b25c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:51:00.720 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:00.416+0000 7effe8ec0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:51:00.720 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:00.547+0000 7effe8ec0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:51:00.721 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:00.641+0000 7effe8ec0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:51:01.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:00.804+0000 7f49b25c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:51:01.062 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:00.756+0000 7effe8ec0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:51:01.062 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:01.061+0000 7effe8ec0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:51:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:01.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:01.782 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:01.399+0000 7f49b25c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:51:01.970 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:01.663+0000 7effe8ec0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:51:01.971 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:51:01 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:51:01.875+0000 7effe8ec0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:51:02.104 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:01.813+0000 7f49b25c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:51:02.105 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:01.907+0000 7f49b25c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:51:02.105 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:02.103+0000 7f49b25c2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:51:02.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:02.156+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f3c980143a0 con 0x7f3ca01087b0 2024-04-25T20:51:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:02 smithi047 ceph-mon[29195]: Standby manager daemon z started 2024-04-25T20:51:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:02 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:51:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:02 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:51:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:02 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:51:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:02 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:51:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:02 smithi186 ceph-mon[32529]: Standby manager daemon z started 2024-04-25T20:51:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:02 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:51:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:02 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:51:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:02 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:51:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:02 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:51:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-mon[32524]: Standby manager daemon z started 2024-04-25T20:51:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:51:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:51:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:51:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.186:0/1861617565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:51:02.946 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:02.620+0000 7f49b25c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:51:02.946 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:02.710+0000 7f49b25c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:02.945+0000 7f49b25c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:03.075+0000 7f49b25c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:03.167+0000 7f49b25c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:03.279+0000 7f49b25c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-mon[32524]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-mon[32524]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3643164498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b824d92-28a7-4a8e-b2fe-1d6512481dea"}]: dispatch 2024-04-25T20:51:03.280 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3643164498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b824d92-28a7-4a8e-b2fe-1d6512481dea"}]': finished 2024-04-25T20:51:03.281 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-mon[32524]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:51:03.281 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:03.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:03 smithi186 ceph-mon[32529]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:03.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:03 smithi186 ceph-mon[32529]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T20:51:03.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:03 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:51:03.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:03 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3643164498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b824d92-28a7-4a8e-b2fe-1d6512481dea"}]: dispatch 2024-04-25T20:51:03.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:03 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3643164498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b824d92-28a7-4a8e-b2fe-1d6512481dea"}]': finished 2024-04-25T20:51:03.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:03 smithi186 ceph-mon[32529]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:51:03.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:03 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:03.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:03 smithi047 ceph-mon[29195]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:03.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:03 smithi047 ceph-mon[29195]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T20:51:03.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:03 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:51:03.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:03 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3643164498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b824d92-28a7-4a8e-b2fe-1d6512481dea"}]: dispatch 2024-04-25T20:51:03.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:03 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3643164498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b824d92-28a7-4a8e-b2fe-1d6512481dea"}]': finished 2024-04-25T20:51:03.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:03 smithi047 ceph-mon[29195]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:51:03.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:03 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:04.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:03.573+0000 7f49b25c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:51:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:04 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2779843346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:04.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:04 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2779843346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:04.533 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:04.156+0000 7f49b25c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:51:04.533 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:51:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:51:04.366+0000 7f49b25c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:51:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:04 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2779843346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:05.170+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f3c98015890 con 0x7f3ca01087b0 2024-04-25T20:51:05.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:05 smithi186 ceph-mon[32529]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:05.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:05 smithi186 ceph-mon[32529]: Standby manager daemon y started 2024-04-25T20:51:05.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:05 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:51:05.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:05 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:51:05.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:05 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:51:05.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:05 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:51:05.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:05 smithi092 ceph-mon[32524]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:05.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:05 smithi092 ceph-mon[32524]: Standby manager daemon y started 2024-04-25T20:51:05.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:05 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:51:05.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:05 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:51:05.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:05 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:51:05.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:05 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:51:05.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:05 smithi047 ceph-mon[29195]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:05.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:05 smithi047 ceph-mon[29195]: Standby manager daemon y started 2024-04-25T20:51:05.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:05 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:51:05.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:05 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:51:05.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:05 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:51:05.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:05 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.92:0/1547103154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:51:06.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:06 smithi186 ceph-mon[32529]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-04-25T20:51:06.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:06 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:51:06.512 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:06 smithi047 ceph-mon[29195]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-04-25T20:51:06.512 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:06 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:51:06.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:06 smithi092 ceph-mon[32524]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-04-25T20:51:06.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:06 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:51:07.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:07 smithi186 ceph-mon[32529]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:07.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:07 smithi092 ceph-mon[32524]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:07.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:07 smithi047 ceph-mon[29195]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:09.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:09 smithi047 ceph-mon[29195]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:09.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:09 smithi186 ceph-mon[32529]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:09.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:09 smithi092 ceph-mon[32524]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:51:10.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:10.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:51:10.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:51:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:11.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:11 smithi047 ceph-mon[29195]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:11.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:11 smithi047 ceph-mon[29195]: Deploying daemon osd.0 on smithi047 2024-04-25T20:51:11.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:11 smithi186 ceph-mon[32529]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:11.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:11 smithi186 ceph-mon[32529]: Deploying daemon osd.0 on smithi047 2024-04-25T20:51:11.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:11 smithi092 ceph-mon[32524]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:11.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:11 smithi092 ceph-mon[32524]: Deploying daemon osd.0 on smithi047 2024-04-25T20:51:13.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:13 smithi186 ceph-mon[32529]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:13.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:13 smithi092 ceph-mon[32524]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:13 smithi047 ceph-mon[29195]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:15.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:15 smithi186 ceph-mon[32529]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:15.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:15 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:15.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:15 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:15.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:15 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:15.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:15 smithi047 ceph-mon[29195]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:15.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:15 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:15.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:15 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:15.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:15 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:15.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:15 smithi092 ceph-mon[32524]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:15.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:15 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:15.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:15 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:15.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:15 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:17.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:17 smithi047 ceph-mon[29195]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:17.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:17 smithi186 ceph-mon[32529]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:17.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:17 smithi092 ceph-mon[32524]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.325+0000 7f3c96ffd700 1 -- 172.21.15.47:0/3239563782 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3ca0062e80 con 0x7f3c8c039890 2024-04-25T20:51:18.326 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 0 on host 'smithi047' 2024-04-25T20:51:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.327+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3c8c039890 msgr2=0x7f3c8c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.327+0000 7f3ca8048700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3c8c039890 0x7f3c8c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b5c0 tx=0x7f3c9c003680 comp rx=0 tx=0).stop 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.327+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca01087b0 msgr2=0x7f3ca01a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.327+0000 7f3ca8048700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca01087b0 0x7f3ca01a23a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c98007f10 tx=0x7f3c9800d3b0 comp rx=0 tx=0).stop 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.327+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 shutdown_connections 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.327+0000 7f3ca8048700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3c8c039890 0x7f3c8c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.328+0000 7f3ca8048700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca01087b0 0x7f3ca01a23a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.328+0000 7f3ca8048700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca0103550 0x7f3ca019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.328+0000 7f3ca8048700 1 --2- 172.21.15.47:0/3239563782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca0105e80 0x7f3ca019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.328+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 >> 172.21.15.47:0/3239563782 conn(0x7f3ca00fcf60 msgr2=0x7f3ca0102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.328+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 shutdown_connections 2024-04-25T20:51:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:51:18.328+0000 7f3ca8048700 1 -- 172.21.15.47:0/3239563782 wait complete. 2024-04-25T20:51:18.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:18.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:18.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:18 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:18.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:18 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:18 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:18 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:18.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:18 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:18.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:18 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:18 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:18 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:18 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:18.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:18 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:18 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:18 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:18.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:18 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:18.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:18 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:18 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:18 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:18.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:18 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:18.962 DEBUG:teuthology.orchestra.run.smithi047:osd.0> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T20:51:18.965 INFO:tasks.cephadm:Deploying osd.1 on smithi092 with /dev/vg_nvme/lv_4... 2024-04-25T20:51:18.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:51:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:51:19.611 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:19 smithi092 ceph-mon[32524]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:19.611 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:19 smithi092 ceph-mon[32524]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:51:19.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:19 smithi047 ceph-mon[29195]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:19.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:19 smithi047 ceph-mon[29195]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:51:19.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:19 smithi186 ceph-mon[32529]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:19.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:19 smithi186 ceph-mon[32529]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:51:20.433 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:20 smithi092 ceph-mon[32524]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:51:20.433 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:20 smithi092 ceph-mon[32524]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:51:20.434 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:20 smithi092 ceph-mon[32524]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:51:20.434 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:20.434 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:20 smithi092 ceph-mon[32524]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:51:20.434 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:20 smithi092 ceph-mon[32524]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:51:20.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:20 smithi047 ceph-mon[29195]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:51:20.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:20 smithi047 ceph-mon[29195]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:51:20.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:20 smithi047 ceph-mon[29195]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:51:20.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:20.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:20 smithi047 ceph-mon[29195]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:51:20.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:20 smithi047 ceph-mon[29195]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:51:20.612 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:51:20 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0[41378]: 2024-04-25T20:51:20.377+0000 7f6d16d11700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T20:51:20.612 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:51:20 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0[41378]: 2024-04-25T20:51:20.383+0000 7f6d12b04700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:51:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:20 smithi186 ceph-mon[32529]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:51:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:20 smithi186 ceph-mon[32529]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:51:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:20 smithi186 ceph-mon[32529]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:51:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:20 smithi186 ceph-mon[32529]: from='osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:51:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:20 smithi186 ceph-mon[32529]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:51:20.968 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:51:20.968 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:51:20.968 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-04-25T20:51:20.968 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-04-25T20:51:20.968 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116014 s, 904 MB/s 2024-04-25T20:51:20.969 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-04-25T20:51:21.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-04-25T20:51:21.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:21 smithi047 ceph-mon[29195]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:21.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:21 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:21.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:21 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:21.715 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:51:21.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:21 smithi186 ceph-mon[32529]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:21.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:21 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:21.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:21 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:21.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:21 smithi092 ceph-mon[32524]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:21.753 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:21 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:21.753 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:21 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:22.701 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] boot 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi047 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.702 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:22.703 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi047 to 17093M 2024-04-25T20:51:22.703 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.703 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:22.703 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:22.703 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] boot 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi047 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:22.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi047 to 17093M 2024-04-25T20:51:22.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:22.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:22.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.839 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:51:22.839 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:51:22.839 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: osd.0 [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] boot 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi047 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi047 to 17093M 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:22.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.132+0000 7f47fb890700 1 -- 172.21.15.92:0/3182010356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 msgr2=0x7f47f41095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.132+0000 7f47fb890700 1 --2- 172.21.15.92:0/3182010356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 0x7f47f41095c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00b0d0 tx=0x7f47ec00b3e0 comp rx=0 tx=0).stop 2024-04-25T20:51:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.132+0000 7f47fb890700 1 -- 172.21.15.92:0/3182010356 shutdown_connections 2024-04-25T20:51:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.132+0000 7f47fb890700 1 --2- 172.21.15.92:0/3182010356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f4101170 0x7f47f4101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.132+0000 7f47fb890700 1 --2- 172.21.15.92:0/3182010356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 0x7f47f41095c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.132+0000 7f47fb890700 1 --2- 172.21.15.92:0/3182010356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47f4102370 0x7f47f41027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.132+0000 7f47fb890700 1 -- 172.21.15.92:0/3182010356 >> 172.21.15.92:0/3182010356 conn(0x7f47f40fc710 msgr2=0x7f47f40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:51:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 -- 172.21.15.92:0/3182010356 shutdown_connections 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 -- 172.21.15.92:0/3182010356 wait complete. 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 Processor -- start 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 -- start start 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47f4101170 0x7f47f4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f4102370 0x7f47f41990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 0x7f47f41a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f47f410d510 con 0x7f47f4102370 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47f410d390 con 0x7f47f4102d10 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47fb890700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f47f410d690 con 0x7f47f4101170 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47f9e2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 0x7f47f41a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47f9e2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 0x7f47f41a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42290/0 (socket says 172.21.15.92:42290) 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47f9e2d700 1 -- 172.21.15.92:0/1588451672 learned_addr learned my addr 172.21.15.92:0/1588451672 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.133+0000 7f47f8e2b700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f4102370 0x7f47f41990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47f962c700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47f4101170 0x7f47f4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47f9e2d700 1 -- 172.21.15.92:0/1588451672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47f4101170 msgr2=0x7f47f4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47f9e2d700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47f4101170 0x7f47f4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47f9e2d700 1 -- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f4102370 msgr2=0x7f47f41990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47f9e2d700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f4102370 0x7f47f41990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47f9e2d700 1 -- 172.21.15.92:0/1588451672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f41a0800 con 0x7f47f4102d10 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47f9e2d700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 0x7f47f41a0160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47ec004c80 tx=0x7f47ec004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47ea7fc700 1 -- 172.21.15.92:0/1588451672 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec01f030 con 0x7f47f4102d10 2024-04-25T20:51:23.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47ea7fc700 1 -- 172.21.15.92:0/1588451672 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ec005040 con 0x7f47f4102d10 2024-04-25T20:51:23.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f41a0a90 con 0x7f47f4102d10 2024-04-25T20:51:23.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47ea7fc700 1 -- 172.21.15.92:0/1588451672 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec01b670 con 0x7f47f4102d10 2024-04-25T20:51:23.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.134+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f41a0f40 con 0x7f47f4102d10 2024-04-25T20:51:23.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.135+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f4069730 con 0x7f47f4102d10 2024-04-25T20:51:23.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.136+0000 7f47ea7fc700 1 -- 172.21.15.92:0/1588451672 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f47ec022030 con 0x7f47f4102d10 2024-04-25T20:51:23.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.137+0000 7f47ea7fc700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47e00a3c30 0x7f47e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:23.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.137+0000 7f47ea7fc700 1 -- 172.21.15.92:0/1588451672 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f47ec00f020 con 0x7f47f4102d10 2024-04-25T20:51:23.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.137+0000 7f47f962c700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47e00a3c30 0x7f47e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:23.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.137+0000 7f47f962c700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47e00a3c30 0x7f47e00a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f47f0005fd0 tx=0x7f47f0005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:51:23.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.142+0000 7f47ea7fc700 1 -- 172.21.15.92:0/1588451672 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47ec029020 con 0x7f47f4102d10 2024-04-25T20:51:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:23.347+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f47f40fe860 con 0x7f47e00a3c30 2024-04-25T20:51:23.955 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:23 smithi092 ceph-mon[32524]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:23.955 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:23 smithi092 ceph-mon[32524]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:51:23.955 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:23.955 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:23.955 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:23.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:23 smithi186 ceph-mon[32529]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:23.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:23 smithi186 ceph-mon[32529]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:51:23.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:23.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:23.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:23 smithi047 ceph-mon[29195]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:51:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:23 smithi047 ceph-mon[29195]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:51:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:24.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:24.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:24.959 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:24 smithi092 ceph-mon[32524]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:24.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:24 smithi186 ceph-mon[32529]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:25.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:24 smithi047 ceph-mon[29195]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:25.890 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:25 smithi186 ceph-mon[32529]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:25 smithi092 ceph-mon[32524]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:25 smithi047 ceph-mon[29195]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:26.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:26 smithi186 ceph-mon[32529]: from='client.? 172.21.15.92:0/3582821065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]: dispatch 2024-04-25T20:51:26.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:26 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]: dispatch 2024-04-25T20:51:26.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:26 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]': finished 2024-04-25T20:51:26.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:26 smithi186 ceph-mon[32529]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:51:26.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:26 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:26.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:26 smithi186 ceph-mon[32529]: from='client.? 172.21.15.92:0/1132205784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:27.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:26 smithi092 ceph-mon[32524]: from='client.? 172.21.15.92:0/3582821065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]: dispatch 2024-04-25T20:51:27.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:26 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]: dispatch 2024-04-25T20:51:27.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:26 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]': finished 2024-04-25T20:51:27.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:26 smithi092 ceph-mon[32524]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:51:27.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:26 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:27.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:26 smithi092 ceph-mon[32524]: from='client.? 172.21.15.92:0/1132205784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:26 smithi047 ceph-mon[29195]: from='client.? 172.21.15.92:0/3582821065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]: dispatch 2024-04-25T20:51:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:26 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]: dispatch 2024-04-25T20:51:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:26 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cef4a330-707b-41e0-bf53-12a1a03da237"}]': finished 2024-04-25T20:51:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:26 smithi047 ceph-mon[29195]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:51:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:26 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:26 smithi047 ceph-mon[29195]: from='client.? 172.21.15.92:0/1132205784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:27.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:27 smithi186 ceph-mon[32529]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:28.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:27 smithi092 ceph-mon[32524]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:27 smithi047 ceph-mon[29195]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:29.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:29 smithi186 ceph-mon[32529]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:29 smithi092 ceph-mon[32524]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:29 smithi047 ceph-mon[29195]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:31.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:31 smithi092 ceph-mon[32524]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:31.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:31 smithi186 ceph-mon[32529]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:32.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:31 smithi047 ceph-mon[29195]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:33.959 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:33 smithi092 ceph-mon[32524]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:33.959 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:51:33.959 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:33.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:33 smithi186 ceph-mon[32529]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:33.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:51:33.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:33 smithi047 ceph-mon[29195]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:51:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:34.745 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:34 smithi092 ceph-mon[32524]: Deploying daemon osd.1 on smithi092 2024-04-25T20:51:34.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:34 smithi186 ceph-mon[32529]: Deploying daemon osd.1 on smithi092 2024-04-25T20:51:35.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:34 smithi047 ceph-mon[29195]: Deploying daemon osd.1 on smithi092 2024-04-25T20:51:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:35 smithi186 ceph-mon[32529]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:36.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:35 smithi047 ceph-mon[29195]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:36.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:35 smithi092 ceph-mon[32524]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:37.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:37 smithi092 ceph-mon[32524]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:37 smithi186 ceph-mon[32529]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:38.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:37 smithi047 ceph-mon[29195]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:39.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:39 smithi186 ceph-mon[32529]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:40.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:39 smithi092 ceph-mon[32524]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:40.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:40.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:40.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:40.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:39 smithi047 ceph-mon[29195]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:40.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:40.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:40.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:41.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:41 smithi092 ceph-mon[32524]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:41.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:41.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:41.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:41.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:41.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:41.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:41 smithi186 ceph-mon[32529]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:41.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:41.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:41.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:41.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:41.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:42.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:41 smithi047 ceph-mon[29195]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:42.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:42.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:42.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:42.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:42.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:42.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.170+0000 7f47ea7fc700 1 -- 172.21.15.92:0/1588451672 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f47f40fe860 con 0x7f47e00a3c30 2024-04-25T20:51:42.171 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 1 on host 'smithi092' 2024-04-25T20:51:42.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47e00a3c30 msgr2=0x7f47e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:42.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47e00a3c30 0x7f47e00a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f47f0005fd0 tx=0x7f47f0005dd0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 msgr2=0x7f47f41a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:42.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 0x7f47f41a0160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47ec004c80 tx=0x7f47ec004cb0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 shutdown_connections 2024-04-25T20:51:42.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47e00a3c30 0x7f47e00a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47f4102d10 0x7f47f41a0160 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47f4101170 0x7f47f4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 --2- 172.21.15.92:0/1588451672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f4102370 0x7f47f41990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 >> 172.21.15.92:0/1588451672 conn(0x7f47f40fc710 msgr2=0x7f47f40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:51:42.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 shutdown_connections 2024-04-25T20:51:42.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:51:42.172+0000 7f47fb890700 1 -- 172.21.15.92:0/1588451672 wait complete. 2024-04-25T20:51:42.741 DEBUG:teuthology.orchestra.run.smithi092:osd.1> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T20:51:42.743 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2024-04-25T20:51:42.743 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:51:42.990 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:51:43.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:43 smithi092 ceph-mon[32524]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:43.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:43.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:43.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:43.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:43 smithi092 ceph-mon[32524]: from='osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:51:43.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:43 smithi092 ceph-mon[32524]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:51:43.404 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:43 smithi186 ceph-mon[32529]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:43.404 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:43.405 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:43.405 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:43.405 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:43 smithi186 ceph-mon[32529]: from='osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:51:43.405 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:43 smithi186 ceph-mon[32529]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:51:43.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:43 smithi047 ceph-mon[29195]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:43.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:43.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:43.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:43.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:43 smithi047 ceph-mon[29195]: from='osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:51:43.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:43 smithi047 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:51:44.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:44 smithi186 ceph-mon[32529]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:51:44.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:44 smithi186 ceph-mon[32529]: from='osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:51:44.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:44 smithi186 ceph-mon[32529]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:51:44.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:44 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:44.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:44 smithi186 ceph-mon[32529]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:51:44.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:44 smithi092 ceph-mon[32524]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:51:44.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:44 smithi092 ceph-mon[32524]: from='osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:51:44.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:44 smithi092 ceph-mon[32524]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:51:44.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:44 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:44.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:44 smithi092 ceph-mon[32524]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:51:44.283 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:51:44 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1[37138]: 2024-04-25T20:51:44.137+0000 7f046e84b700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T20:51:44.284 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:51:44 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1[37138]: 2024-04-25T20:51:44.141+0000 7f046a63e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:51:44.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:44 smithi047 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:51:44.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:44 smithi047 ceph-mon[29195]: from='osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:51:44.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:44 smithi047 ceph-mon[29195]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:51:44.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:44 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:44.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:44 smithi047 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:51:44.696 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:51:44.696 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:51:44.696 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-25T20:51:44.696 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-25T20:51:44.696 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112988 s, 928 MB/s 2024-04-25T20:51:44.696 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-25T20:51:45.269 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2024-04-25T20:51:45.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:45 smithi092 ceph-mon[32524]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:45.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:45 smithi092 ceph-mon[32524]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:51:45.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:45 smithi092 ceph-mon[32524]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:51:45.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:45.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:45.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:45 smithi186 ceph-mon[32529]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:45.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:45 smithi186 ceph-mon[32529]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:51:45.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:45 smithi186 ceph-mon[32529]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:51:45.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:45.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:45.515 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:51:45.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:45 smithi047 ceph-mon[29195]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:51:45.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:45 smithi047 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:51:45.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:45 smithi047 ceph-mon[29195]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:51:45.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:45.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] boot 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi092 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi092 to 17088M 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.395 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:46.396 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:46.396 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] boot 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi092 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi092 to 17088M 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:51:46.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:51:46.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: osd.1 [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] boot 2024-04-25T20:51:46.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:51:46.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi092 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi092 to 17088M 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.943+0000 7f8f67c52700 1 -- 172.21.15.186:0/1077111529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60103540 msgr2=0x7f8f60109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.943+0000 7f8f67c52700 1 --2- 172.21.15.186:0/1077111529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60103540 0x7f8f60109df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800b0a0 tx=0x7f8f5800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.943+0000 7f8f67c52700 1 -- 172.21.15.186:0/1077111529 shutdown_connections 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.943+0000 7f8f67c52700 1 --2- 172.21.15.186:0/1077111529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f601019a0 0x7f8f60101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.943+0000 7f8f67c52700 1 --2- 172.21.15.186:0/1077111529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60103540 0x7f8f60109df0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.943+0000 7f8f67c52700 1 --2- 172.21.15.186:0/1077111529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f60102ba0 0x7f8f60103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.943+0000 7f8f67c52700 1 -- 172.21.15.186:0/1077111529 >> 172.21.15.186:0/1077111529 conn(0x7f8f600fcf40 msgr2=0x7f8f600ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.943+0000 7f8f67c52700 1 -- 172.21.15.186:0/1077111529 shutdown_connections 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 -- 172.21.15.186:0/1077111529 wait complete. 2024-04-25T20:51:46.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 Processor -- start 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 -- start start 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f601019a0 0x7f8f60198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60102ba0 0x7f8f60199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f60103540 0x7f8f601a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6010d440 con 0x7f8f60103540 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6010d2c0 con 0x7f8f601019a0 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f67c52700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6010d5c0 con 0x7f8f60102ba0 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f651ed700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60102ba0 0x7f8f60199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f651ed700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60102ba0 0x7f8f60199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:53854/0 (socket says 172.21.15.186:53854) 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f651ed700 1 -- 172.21.15.186:0/2334659808 learned_addr learned my addr 172.21.15.186:0/2334659808 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.944+0000 7f8f659ee700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f601019a0 0x7f8f60198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f661ef700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f60103540 0x7f8f601a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f651ed700 1 -- 172.21.15.186:0/2334659808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f601019a0 msgr2=0x7f8f60198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:46.948 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f651ed700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f601019a0 0x7f8f60198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f651ed700 1 -- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f60103540 msgr2=0x7f8f601a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f651ed700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f60103540 0x7f8f601a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f651ed700 1 -- 172.21.15.186:0/2334659808 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f601a07f0 con 0x7f8f60102ba0 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f661ef700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f60103540 0x7f8f601a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f651ed700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60102ba0 0x7f8f60199120 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000c9a0 tx=0x7f8f5000ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f56ffd700 1 -- 172.21.15.186:0/2334659808 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50007a60 con 0x7f8f60102ba0 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f56ffd700 1 -- 172.21.15.186:0/2334659808 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f50004ce0 con 0x7f8f60102ba0 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f601a0ae0 con 0x7f8f60102ba0 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f56ffd700 1 -- 172.21.15.186:0/2334659808 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50005560 con 0x7f8f60102ba0 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.945+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f601a0fd0 con 0x7f8f60102ba0 2024-04-25T20:51:46.949 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.946+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f60069730 con 0x7f8f60102ba0 2024-04-25T20:51:46.950 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.947+0000 7f8f56ffd700 1 -- 172.21.15.186:0/2334659808 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f8f50007c00 con 0x7f8f60102ba0 2024-04-25T20:51:46.950 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.948+0000 7f8f56ffd700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8f4c0a3bb0 0x7f8f4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:46.950 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.948+0000 7f8f56ffd700 1 -- 172.21.15.186:0/2334659808 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f8f500ce880 con 0x7f8f60102ba0 2024-04-25T20:51:46.950 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.948+0000 7f8f659ee700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8f4c0a3bb0 0x7f8f4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:46.950 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.949+0000 7f8f659ee700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8f4c0a3bb0 0x7f8f4c0a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f60102a00 tx=0x7f8f5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:51:46.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:46.953+0000 7f8f56ffd700 1 -- 172.21.15.186:0/2334659808 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f5009cb40 con 0x7f8f60102ba0 2024-04-25T20:51:47.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:51:47.156+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8f600ff150 con 0x7f8f4c0a3bb0 2024-04-25T20:51:47.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:47 smithi186 ceph-mon[32529]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:47.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:47 smithi186 ceph-mon[32529]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:51:47.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:47.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:47.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:47.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:47 smithi092 ceph-mon[32524]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:47 smithi092 ceph-mon[32524]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:51:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:47.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:47 smithi047 ceph-mon[29195]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:47.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:47 smithi047 ceph-mon[29195]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:51:47.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:47.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:47.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:48.755 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:48 smithi186 ceph-mon[32529]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:48 smithi092 ceph-mon[32524]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:48.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:48 smithi047 ceph-mon[29195]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:49.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:49 smithi092 ceph-mon[32524]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:49.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:49 smithi047 ceph-mon[29195]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:49.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:49 smithi186 ceph-mon[32529]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:50.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:50 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/26129263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]: dispatch 2024-04-25T20:51:50.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:50 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]: dispatch 2024-04-25T20:51:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:50 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]': finished 2024-04-25T20:51:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:50 smithi092 ceph-mon[32524]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:51:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:51:50.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:50 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/26129263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]: dispatch 2024-04-25T20:51:50.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:50 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]: dispatch 2024-04-25T20:51:50.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:50 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]': finished 2024-04-25T20:51:50.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:50 smithi047 ceph-mon[29195]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:51:50.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:51:50.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:50 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/26129263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]: dispatch 2024-04-25T20:51:50.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:50 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]: dispatch 2024-04-25T20:51:50.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:50 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49a4bac4-8c22-47d9-b96f-56e99bd8cec0"}]': finished 2024-04-25T20:51:50.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:50 smithi186 ceph-mon[32529]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:51:50.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:51:51.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:51 smithi092 ceph-mon[32524]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:51.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:51 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/2154773006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:51 smithi047 ceph-mon[29195]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:51 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/2154773006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:51.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:51 smithi186 ceph-mon[32529]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:51.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:51 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/2154773006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:53.755 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:53 smithi186 ceph-mon[32529]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:53.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:53 smithi092 ceph-mon[32524]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:53 smithi047 ceph-mon[29195]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:55.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:55 smithi092 ceph-mon[32524]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:55.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:55 smithi186 ceph-mon[32529]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:55.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:55 smithi047 ceph-mon[29195]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:57 smithi186 ceph-mon[32529]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:57 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:51:57.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:57 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:57.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:57 smithi092 ceph-mon[32524]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:57.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:57 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:51:57.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:57 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:57.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:57 smithi047 ceph-mon[29195]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:57.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:51:57.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:58.700 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:58 smithi186 ceph-mon[32529]: Deploying daemon osd.2 on smithi186 2024-04-25T20:51:58.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:58 smithi092 ceph-mon[32524]: Deploying daemon osd.2 on smithi186 2024-04-25T20:51:58.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:58 smithi047 ceph-mon[29195]: Deploying daemon osd.2 on smithi186 2024-04-25T20:51:59.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:51:59 smithi186 ceph-mon[32529]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:59.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:51:59 smithi092 ceph-mon[32524]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:51:59.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:51:59 smithi047 ceph-mon[29195]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:01.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:01 smithi186 ceph-mon[32529]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:01.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:01 smithi092 ceph-mon[32524]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:01 smithi047 ceph-mon[29195]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:03 smithi186 ceph-mon[32529]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:03 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:03 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:03 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:03.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:03 smithi047 ceph-mon[29195]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:03.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:03 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:03.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:03 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:03.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:03 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:04.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:03 smithi092 ceph-mon[32524]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:04.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:03 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:04.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:03 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:04.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:03 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:05.708 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:05 smithi186 ceph-mon[32529]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:05.708 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:05.709 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:05.709 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:05.709 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:05.709 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:05 smithi047 ceph-mon[29195]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:05 smithi092 ceph-mon[32524]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:06.326 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2024-04-25T20:52:06.326 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.324+0000 7f8f56ffd700 1 -- 172.21.15.186:0/2334659808 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f600ff150 con 0x7f8f4c0a3bb0 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8f4c0a3bb0 msgr2=0x7f8f4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8f4c0a3bb0 0x7f8f4c0a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f60102a00 tx=0x7f8f5c00b410 comp rx=0 tx=0).stop 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60102ba0 msgr2=0x7f8f60199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60102ba0 0x7f8f60199120 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000c9a0 tx=0x7f8f5000ccb0 comp rx=0 tx=0).stop 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 shutdown_connections 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8f4c0a3bb0 0x7f8f4c0a6070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f60103540 0x7f8f601a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f601019a0 0x7f8f60198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 --2- 172.21.15.186:0/2334659808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f60102ba0 0x7f8f60199120 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 >> 172.21.15.186:0/2334659808 conn(0x7f8f600fcf40 msgr2=0x7f8f600fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 shutdown_connections 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:06.327+0000 7f8f67c52700 1 -- 172.21.15.186:0/2334659808 wait complete. 2024-04-25T20:52:06.975 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T20:52:06.977 INFO:tasks.cephadm:Deploying osd.3 on smithi047 with /dev/vg_nvme/lv_3... 2024-04-25T20:52:06.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T20:52:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:52:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:07 smithi186 ceph-mon[32529]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:07 smithi186 ceph-mon[32529]: from='osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:52:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:07 smithi186 ceph-mon[32529]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:07 smithi047 ceph-mon[29195]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:07.586 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:07.586 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:07.586 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:07 smithi047 ceph-mon[29195]: from='osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:52:07.586 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:07 smithi047 ceph-mon[29195]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:52:07.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:07 smithi092 ceph-mon[32524]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:07 smithi092 ceph-mon[32524]: from='osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:52:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:07 smithi092 ceph-mon[32524]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:52:08.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:08 smithi186 ceph-mon[32529]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:52:08.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:08 smithi186 ceph-mon[32529]: from='osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:52:08.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:08 smithi186 ceph-mon[32529]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:52:08.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:08 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:08.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:08 smithi186 ceph-mon[32529]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:52:08.471 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2[37727]: 2024-04-25T20:52:08.330+0000 7f71ba4a5700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T20:52:08.471 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2[37727]: 2024-04-25T20:52:08.334+0000 7f71b6298700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:52:08.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:08 smithi047 ceph-mon[29195]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:52:08.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:08 smithi047 ceph-mon[29195]: from='osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:52:08.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:08 smithi047 ceph-mon[29195]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:52:08.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:08 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:08.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:08 smithi047 ceph-mon[29195]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:52:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:08 smithi092 ceph-mon[32524]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:52:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:08 smithi092 ceph-mon[32524]: from='osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:52:08.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:08 smithi092 ceph-mon[32524]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:52:08.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:08 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:08.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:08 smithi092 ceph-mon[32524]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:52:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T20:52:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T20:52:09.006 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-04-25T20:52:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-04-25T20:52:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00908881 s, 1.2 GB/s 2024-04-25T20:52:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-04-25T20:52:09.556 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-04-25T20:52:09.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:09 smithi047 ceph-mon[29195]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:09.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:09 smithi047 ceph-mon[29195]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:52:09.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:09 smithi047 ceph-mon[29195]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:52:09.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:09 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:09.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:09 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:09.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:09 smithi186 ceph-mon[32529]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:09.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:09 smithi186 ceph-mon[32529]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:52:09.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:09 smithi186 ceph-mon[32529]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:52:09.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:09 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:09.652 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:09 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:09.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:09 smithi092 ceph-mon[32524]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:09.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:09 smithi092 ceph-mon[32524]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:52:09.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:09 smithi092 ceph-mon[32524]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:52:09.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:09 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:09.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:09 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:09.805 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:52:10.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:52:10.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] boot 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] boot 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:10.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:10.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:52:10.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: osd.2 [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] boot 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:52:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 -- 172.21.15.47:0/3751938442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090102ba0 msgr2=0x7fe090103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 --2- 172.21.15.47:0/3751938442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090102ba0 0x7fe090103000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0780099c0 tx=0x7fe078009cd0 comp rx=0 tx=0).stop 2024-04-25T20:52:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 -- 172.21.15.47:0/3751938442 shutdown_connections 2024-04-25T20:52:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 --2- 172.21.15.47:0/3751938442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe090103540 0x7fe090109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 --2- 172.21.15.47:0/3751938442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0901019a0 0x7fe090101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 --2- 172.21.15.47:0/3751938442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090102ba0 0x7fe090103000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 -- 172.21.15.47:0/3751938442 >> 172.21.15.47:0/3751938442 conn(0x7fe0900fcf40 msgr2=0x7fe0900ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 -- 172.21.15.47:0/3751938442 shutdown_connections 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.333+0000 7fe09617c700 1 -- 172.21.15.47:0/3751938442 wait complete. 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.334+0000 7fe09617c700 1 Processor -- start 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.334+0000 7fe09617c700 1 -- start start 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.334+0000 7fe09617c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0901019a0 0x7fe09019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.334+0000 7fe09617c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe090102ba0 0x7fe09019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.334+0000 7fe09617c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090103540 0x7fe0901a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.334+0000 7fe09617c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe09010d430 con 0x7fe090103540 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.334+0000 7fe09617c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe09010d2b0 con 0x7fe090102ba0 2024-04-25T20:52:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.334+0000 7fe09617c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe09010d5b0 con 0x7fe0901019a0 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090103540 0x7fe0901a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090103540 0x7fe0901a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52702/0 (socket says 172.21.15.47:52702) 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 -- 172.21.15.47:0/549921437 learned_addr learned my addr 172.21.15.47:0/549921437 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08f7fe700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0901019a0 0x7fe09019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe086ffd700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe090102ba0 0x7fe09019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 -- 172.21.15.47:0/549921437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0901019a0 msgr2=0x7fe09019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0901019a0 0x7fe09019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 -- 172.21.15.47:0/549921437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe090102ba0 msgr2=0x7fe09019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe090102ba0 0x7fe09019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 -- 172.21.15.47:0/549921437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0901a2a80 con 0x7fe090103540 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08ffff700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090103540 0x7fe0901a23e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe088007f10 tx=0x7fe08800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08d7fa700 1 -- 172.21.15.47:0/549921437 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088004bb0 con 0x7fe090103540 2024-04-25T20:52:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0901a2d70 con 0x7fe090103540 2024-04-25T20:52:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08d7fa700 1 -- 172.21.15.47:0/549921437 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe088004d50 con 0x7fe090103540 2024-04-25T20:52:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0901a3200 con 0x7fe090103540 2024-04-25T20:52:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.335+0000 7fe08d7fa700 1 -- 172.21.15.47:0/549921437 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0880058e0 con 0x7fe090103540 2024-04-25T20:52:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.336+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe070005360 con 0x7fe090103540 2024-04-25T20:52:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.338+0000 7fe08d7fa700 1 -- 172.21.15.47:0/549921437 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fe08801a020 con 0x7fe090103540 2024-04-25T20:52:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.338+0000 7fe08d7fa700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe07c0a3bb0 0x7fe07c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.338+0000 7fe08f7fe700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe07c0a3bb0 0x7fe07c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.338+0000 7fe08d7fa700 1 -- 172.21.15.47:0/549921437 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fe0880ce540 con 0x7fe090103540 2024-04-25T20:52:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.339+0000 7fe08f7fe700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe07c0a3bb0 0x7fe07c0a6070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe090102a00 tx=0x7fe080009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.343+0000 7fe08d7fa700 1 -- 172.21.15.47:0/549921437 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe088010070 con 0x7fe090103540 2024-04-25T20:52:11.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:11.579+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe070000bf0 con 0x7fe07c0a3bb0 2024-04-25T20:52:11.607 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41473]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:52:11.608 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41473]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:11.608 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41473]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:11.608 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41473]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:11.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi186 2024-04-25T20:52:11.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-25T20:52:11.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 ceph-mon[32529]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:11.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T20:52:11.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 ceph-mon[32529]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T20:52:11.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:52:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi186 2024-04-25T20:52:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-25T20:52:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 ceph-mon[29195]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T20:52:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 ceph-mon[29195]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T20:52:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:52:11.611 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45623]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:52:11.611 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:11.612 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45623]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:11.612 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45623]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:11.612 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:52:11.612 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:11.612 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45626]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:11.612 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45626]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:11.719 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi186 2024-04-25T20:52:11.719 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-25T20:52:11.719 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 ceph-mon[32524]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:52:11.719 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T20:52:11.719 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 ceph-mon[32524]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T20:52:11.719 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:52:11.719 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:52:11.719 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:11.719 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40818]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:11.719 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40818]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:11.720 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:52:11.720 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:11.720 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40821]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:11.720 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40821]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:11.970 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41476]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:52:11.971 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41476]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:11.971 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41476]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:11.971 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41476]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:52:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41479]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41479]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:52:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:11 smithi186 sudo[41482]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:12.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:52:12.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:12.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40824]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:12.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40824]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:12.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40827]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:52:12.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40827]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:12.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40827]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:12.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:11 smithi092 sudo[40827]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45680]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:52:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45680]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45680]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45680]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45699]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:52:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45699]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:52:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45699]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:52:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:11 smithi047 sudo[45699]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:52:12.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:12 smithi047 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:12.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:12.642+0000 7fe08d7fa700 1 -- 172.21.15.47:0/549921437 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe088098350 con 0x7fe090103540 2024-04-25T20:52:12.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T20:52:12.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T20:52:12.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:12.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:12.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:12.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:12 smithi186 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:52:12.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:12 smithi092 ceph-mon[32524]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:52:13.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:13 smithi047 ceph-mon[29195]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:13.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:13 smithi047 ceph-mon[29195]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T20:52:13.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:13 smithi047 ceph-mon[29195]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T20:52:13.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:13 smithi186 ceph-mon[32529]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:13.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:13 smithi186 ceph-mon[32529]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T20:52:13.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:13 smithi186 ceph-mon[32529]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T20:52:13.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:13 smithi092 ceph-mon[32524]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:13.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:13 smithi092 ceph-mon[32524]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T20:52:13.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:13 smithi092 ceph-mon[32524]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T20:52:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:14 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2175550827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f303ea7-0cef-436c-82ed-e3a7f1fde8ab"}]: dispatch 2024-04-25T20:52:14.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:14 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2175550827' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f303ea7-0cef-436c-82ed-e3a7f1fde8ab"}]': finished 2024-04-25T20:52:14.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:14 smithi186 ceph-mon[32529]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T20:52:14.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:14 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:14.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:14 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2175550827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f303ea7-0cef-436c-82ed-e3a7f1fde8ab"}]: dispatch 2024-04-25T20:52:14.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:14 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2175550827' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f303ea7-0cef-436c-82ed-e3a7f1fde8ab"}]': finished 2024-04-25T20:52:14.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:14 smithi092 ceph-mon[32524]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T20:52:14.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:14 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:14.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:14 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2175550827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f303ea7-0cef-436c-82ed-e3a7f1fde8ab"}]: dispatch 2024-04-25T20:52:14.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:14 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2175550827' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f303ea7-0cef-436c-82ed-e3a7f1fde8ab"}]': finished 2024-04-25T20:52:14.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:14 smithi047 ceph-mon[29195]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T20:52:14.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:14 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:15 smithi186 ceph-mon[32529]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:15 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3902229110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:15.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:15 smithi092 ceph-mon[32524]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:15.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:15 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3902229110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:15 smithi047 ceph-mon[29195]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:15 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3902229110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:17.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:17 smithi186 ceph-mon[32529]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:17.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:17 smithi092 ceph-mon[32524]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:17 smithi047 ceph-mon[29195]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:19.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:19 smithi186 ceph-mon[32529]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:19.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:19 smithi092 ceph-mon[32524]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:19.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:19 smithi047 ceph-mon[29195]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:21.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:21 smithi186 ceph-mon[32529]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:21.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:21 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T20:52:21.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:21 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:21.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:21 smithi092 ceph-mon[32524]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:21.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:21 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T20:52:21.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:21 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:21.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:21 smithi047 ceph-mon[29195]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:21.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:21 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T20:52:21.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:21 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:22.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:22 smithi047 ceph-mon[29195]: Deploying daemon osd.3 on smithi047 2024-04-25T20:52:22.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:22 smithi186 ceph-mon[32529]: Deploying daemon osd.3 on smithi047 2024-04-25T20:52:22.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:22 smithi092 ceph-mon[32524]: Deploying daemon osd.3 on smithi047 2024-04-25T20:52:23.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:23 smithi047 ceph-mon[29195]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:23.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:23 smithi186 ceph-mon[32529]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:23.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:23 smithi092 ceph-mon[32524]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:25.573 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:25 smithi047 ceph-mon[29195]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:25.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:25 smithi186 ceph-mon[32529]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:25.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:25 smithi092 ceph-mon[32524]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:27.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:27 smithi092 ceph-mon[32524]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:27.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:27 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:27.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:27 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:27.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:27 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:27 smithi047 ceph-mon[29195]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:27 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:27 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:27 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:27.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:27 smithi186 ceph-mon[32529]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:27.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:27 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:27.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:27 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:27.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:27 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:29.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:29 smithi047 ceph-mon[29195]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:29.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:29.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:29.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:29.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:29.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:29.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.683+0000 7fe08d7fa700 1 -- 172.21.15.47:0/549921437 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe070000bf0 con 0x7fe07c0a3bb0 2024-04-25T20:52:29.685 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 3 on host 'smithi047' 2024-04-25T20:52:29.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.685+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe07c0a3bb0 msgr2=0x7fe07c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:29.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.685+0000 7fe09617c700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe07c0a3bb0 0x7fe07c0a6070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe090102a00 tx=0x7fe080009260 comp rx=0 tx=0).stop 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.685+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090103540 msgr2=0x7fe0901a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.685+0000 7fe09617c700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090103540 0x7fe0901a23e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe088007f10 tx=0x7fe08800d3b0 comp rx=0 tx=0).stop 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.686+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 shutdown_connections 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.686+0000 7fe09617c700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe07c0a3bb0 0x7fe07c0a6070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.686+0000 7fe09617c700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe090103540 0x7fe0901a23e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.686+0000 7fe09617c700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0901019a0 0x7fe09019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.686+0000 7fe09617c700 1 --2- 172.21.15.47:0/549921437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe090102ba0 0x7fe09019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.686+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 >> 172.21.15.47:0/549921437 conn(0x7fe0900fcf40 msgr2=0x7fe09006a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.686+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 shutdown_connections 2024-04-25T20:52:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:52:29.686+0000 7fe09617c700 1 -- 172.21.15.47:0/549921437 wait complete. 2024-04-25T20:52:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:29 smithi186 ceph-mon[32529]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:29 smithi092 ceph-mon[32524]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.265 DEBUG:teuthology.orchestra.run.smithi047:osd.3> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T20:52:30.268 INFO:tasks.cephadm:Deploying osd.4 on smithi092 with /dev/vg_nvme/lv_3... 2024-04-25T20:52:30.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T20:52:30.511 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:52:30.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:30 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:30 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:30 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:30.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:30 smithi047 ceph-mon[29195]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T20:52:30.949 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:30 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.949 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:30 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.949 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:30 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:30.949 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:30 smithi092 ceph-mon[32524]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T20:52:30.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:30 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:30 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:30.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:30 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:30.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:30 smithi186 ceph-mon[32529]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T20:52:31.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:31 smithi092 ceph-mon[32524]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:31.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:31 smithi092 ceph-mon[32524]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T20:52:31.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:31 smithi092 ceph-mon[32524]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T20:52:31.872 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:31 smithi092 ceph-mon[32524]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:52:31.872 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:31 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:31.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:31 smithi186 ceph-mon[32529]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:31.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:31 smithi186 ceph-mon[32529]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T20:52:31.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:31 smithi186 ceph-mon[32529]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T20:52:31.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:31 smithi186 ceph-mon[32529]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:52:31.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:31 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:32.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:31 smithi047 ceph-mon[29195]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:32.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:31 smithi047 ceph-mon[29195]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T20:52:32.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:31 smithi047 ceph-mon[29195]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T20:52:32.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:31 smithi047 ceph-mon[29195]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:52:32.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:31 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:32.112 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 20:52:31 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3[47878]: 2024-04-25T20:52:31.683+0000 7f2d8d65f700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T20:52:32.112 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 20:52:31 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3[47878]: 2024-04-25T20:52:31.689+0000 7f2d89452700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:52:32.441 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T20:52:32.441 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T20:52:32.441 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-04-25T20:52:32.441 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-04-25T20:52:32.442 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115004 s, 912 MB/s 2024-04-25T20:52:32.442 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-04-25T20:52:32.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:32 smithi186 ceph-mon[32529]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:52:32.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:32 smithi186 ceph-mon[32529]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T20:52:32.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:32.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:32 smithi092 ceph-mon[32524]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:52:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:32 smithi092 ceph-mon[32524]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T20:52:33.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-04-25T20:52:33.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:32 smithi047 ceph-mon[29195]: from='osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:52:33.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:32 smithi047 ceph-mon[29195]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T20:52:33.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.343 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] boot 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:33.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:33.710 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.710 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.710 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:33.710 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:33.710 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:33.710 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] boot 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:33.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:33.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:33.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:52:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:52:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:52:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: osd.3 [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] boot 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:34.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:34.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:34 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi047 2024-04-25T20:52:34.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:34 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi047 to 8546M 2024-04-25T20:52:34.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:34 smithi092 ceph-mon[32524]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T20:52:34.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 -- 172.21.15.92:0/4004086779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c104640 msgr2=0x7f5f7c10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:34.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 --2- 172.21.15.92:0/4004086779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c104640 0x7f5f7c10aef0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400b0d0 tx=0x7f5f7400b3e0 comp rx=0 tx=0).stop 2024-04-25T20:52:34.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 -- 172.21.15.92:0/4004086779 shutdown_connections 2024-04-25T20:52:34.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 --2- 172.21.15.92:0/4004086779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c104640 0x7f5f7c10aef0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:34.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 --2- 172.21.15.92:0/4004086779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c102aa0 0x7f5f7c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:34.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 --2- 172.21.15.92:0/4004086779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c103ca0 0x7f5f7c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:34.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 -- 172.21.15.92:0/4004086779 >> 172.21.15.92:0/4004086779 conn(0x7f5f7c0fe040 msgr2=0x7f5f7c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 -- 172.21.15.92:0/4004086779 shutdown_connections 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 -- 172.21.15.92:0/4004086779 wait complete. 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.835+0000 7f5f8135c700 1 Processor -- start 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f8135c700 1 -- start start 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f8135c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c102aa0 0x7f5f7c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f8135c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c103ca0 0x7f5f7c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f8135c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c104640 0x7f5f7c1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f8135c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10e540 con 0x7f5f7c104640 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f8135c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10e3c0 con 0x7f5f7c102aa0 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f8135c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10e6c0 con 0x7f5f7c103ca0 2024-04-25T20:52:34.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c102aa0 0x7f5f7c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c102aa0 0x7f5f7c198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33862/0 (socket says 172.21.15.92:33862) 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 -- 172.21.15.92:0/1547791370 learned_addr learned my addr 172.21.15.92:0/1547791370 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c103ca0 0x7f5f7c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7b7fe700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c104640 0x7f5f7c1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 -- 172.21.15.92:0/1547791370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c103ca0 msgr2=0x7f5f7c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c103ca0 0x7f5f7c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 -- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c104640 msgr2=0x7f5f7c1a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c104640 0x7f5f7c1a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 -- 172.21.15.92:0/1547791370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c1a07e0 con 0x7f5f7c102aa0 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f7affd700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c102aa0 0x7f5f7c198b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00c8f0 tx=0x7f5f6c00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f63fff700 1 -- 172.21.15.92:0/1547791370 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004220 con 0x7f5f7c102aa0 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7c1a0ad0 con 0x7f5f7c102aa0 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f63fff700 1 -- 172.21.15.92:0/1547791370 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6c007db0 con 0x7f5f7c102aa0 2024-04-25T20:52:34.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.836+0000 7f5f63fff700 1 -- 172.21.15.92:0/1547791370 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c003d00 con 0x7f5f7c102aa0 2024-04-25T20:52:34.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.837+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c1a1010 con 0x7f5f7c102aa0 2024-04-25T20:52:34.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.838+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f7c069730 con 0x7f5f7c102aa0 2024-04-25T20:52:34.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.839+0000 7f5f63fff700 1 -- 172.21.15.92:0/1547791370 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5f6c007900 con 0x7f5f7c102aa0 2024-04-25T20:52:34.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.839+0000 7f5f63fff700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f640a3c40 0x7f5f640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:34.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.839+0000 7f5f63fff700 1 -- 172.21.15.92:0/1547791370 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f5f6c0cee50 con 0x7f5f7c102aa0 2024-04-25T20:52:34.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.839+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f640a3c40 0x7f5f640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:34.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.840+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f640a3c40 0x7f5f640a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5f70005fd0 tx=0x7f5f70005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:34.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:34.845+0000 7f5f63fff700 1 -- 172.21.15.92:0/1547791370 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f6c014030 con 0x7f5f7c102aa0 2024-04-25T20:52:34.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:34 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi047 2024-04-25T20:52:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:34 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi047 to 8546M 2024-04-25T20:52:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:34 smithi186 ceph-mon[32529]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T20:52:35.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:35.053+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5f7c062e80 con 0x7f5f640a3c40 2024-04-25T20:52:35.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:34 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi047 2024-04-25T20:52:35.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:34 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi047 to 8546M 2024-04-25T20:52:35.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:34 smithi047 ceph-mon[29195]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T20:52:35.929 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:35 smithi092 ceph-mon[32524]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:35.929 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:35 smithi092 ceph-mon[32524]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T20:52:35.929 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:35 smithi092 ceph-mon[32524]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:35.929 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:35.929 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:35.929 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:35 smithi186 ceph-mon[32529]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:35 smithi186 ceph-mon[32529]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T20:52:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:35 smithi186 ceph-mon[32529]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:36.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:35 smithi047 ceph-mon[29195]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:36.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:35 smithi047 ceph-mon[29195]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T20:52:36.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:35 smithi047 ceph-mon[29195]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:36.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:36.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:36.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:36.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:36 smithi092 ceph-mon[32524]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T20:52:36.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:36 smithi092 ceph-mon[32524]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:36 smithi186 ceph-mon[32529]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T20:52:36.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:36 smithi186 ceph-mon[32529]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:37.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:36 smithi047 ceph-mon[29195]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T20:52:37.013 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:36 smithi047 ceph-mon[29195]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:37 smithi186 ceph-mon[32529]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T20:52:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:37 smithi186 ceph-mon[32529]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T20:52:37.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:37 smithi186 ceph-mon[32529]: from='client.? 172.21.15.92:0/3811123740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]: dispatch 2024-04-25T20:52:37.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:37 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]: dispatch 2024-04-25T20:52:37.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:37 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]': finished 2024-04-25T20:52:37.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:37 smithi186 ceph-mon[32529]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T20:52:37.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:37 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:38.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:37 smithi047 ceph-mon[29195]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T20:52:38.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:37 smithi047 ceph-mon[29195]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T20:52:38.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:37 smithi047 ceph-mon[29195]: from='client.? 172.21.15.92:0/3811123740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]: dispatch 2024-04-25T20:52:38.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:37 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]: dispatch 2024-04-25T20:52:38.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:37 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]': finished 2024-04-25T20:52:38.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:37 smithi047 ceph-mon[29195]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T20:52:38.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:37 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:37 smithi092 ceph-mon[32524]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T20:52:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:37 smithi092 ceph-mon[32524]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T20:52:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:37 smithi092 ceph-mon[32524]: from='client.? 172.21.15.92:0/3811123740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]: dispatch 2024-04-25T20:52:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:37 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]: dispatch 2024-04-25T20:52:38.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:37 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05"}]': finished 2024-04-25T20:52:38.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:37 smithi092 ceph-mon[32524]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T20:52:38.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:37 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:39.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:38 smithi092 ceph-mon[32524]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:39.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:38 smithi092 ceph-mon[32524]: from='client.? 172.21.15.92:0/1856174153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:39.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:38 smithi047 ceph-mon[29195]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:39.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:38 smithi047 ceph-mon[29195]: from='client.? 172.21.15.92:0/1856174153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:38 smithi186 ceph-mon[32529]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:38 smithi186 ceph-mon[32529]: from='client.? 172.21.15.92:0/1856174153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:41.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:40 smithi186 ceph-mon[32529]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:41.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:40 smithi092 ceph-mon[32524]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:41.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:40 smithi047 ceph-mon[29195]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T20:52:42.180 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:41 smithi092 ceph-mon[32524]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T20:52:42.180 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:41 smithi092 ceph-mon[32524]: Cluster is now healthy 2024-04-25T20:52:42.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:41 smithi186 ceph-mon[32529]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T20:52:42.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:41 smithi186 ceph-mon[32529]: Cluster is now healthy 2024-04-25T20:52:42.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:41 smithi047 ceph-mon[29195]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T20:52:42.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:41 smithi047 ceph-mon[29195]: Cluster is now healthy 2024-04-25T20:52:43.135 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:42 smithi092 ceph-mon[32524]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T20:52:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:42 smithi186 ceph-mon[32529]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T20:52:43.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:42 smithi047 ceph-mon[29195]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T20:52:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:44 smithi186 ceph-mon[32529]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T20:52:45.301 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:44 smithi092 ceph-mon[32524]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T20:52:45.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:44 smithi047 ceph-mon[29195]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T20:52:46.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T20:52:46.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:45 smithi186 ceph-mon[32529]: Deploying daemon osd.4 on smithi092 2024-04-25T20:52:46.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T20:52:46.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:46.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:45 smithi092 ceph-mon[32524]: Deploying daemon osd.4 on smithi092 2024-04-25T20:52:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T20:52:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:45 smithi047 ceph-mon[29195]: Deploying daemon osd.4 on smithi092 2024-04-25T20:52:47.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:46 smithi186 ceph-mon[32529]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T20:52:47.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:46 smithi092 ceph-mon[32524]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T20:52:47.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:46 smithi047 ceph-mon[29195]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T20:52:49.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:48 smithi047 ceph-mon[29195]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T20:52:49.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:48 smithi186 ceph-mon[32529]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T20:52:49.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:48 smithi092 ceph-mon[32524]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T20:52:51.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:51 smithi186 ceph-mon[32529]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T20:52:51.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:51 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:51.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:51 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:51.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:51 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:51.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:51 smithi092 ceph-mon[32524]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T20:52:51.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:51 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:51.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:51 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:51.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:51 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:51 smithi047 ceph-mon[29195]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T20:52:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:53.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:53 smithi186 ceph-mon[32529]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T20:52:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:53.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:53 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:53.732 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:53 smithi092 ceph-mon[32524]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T20:52:53.732 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:53.733 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:53.733 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:53.733 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:53.733 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:53 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:53 smithi047 ceph-mon[29195]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T20:52:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:53 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:54.071 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 4 on host 'smithi092' 2024-04-25T20:52:54.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.070+0000 7f5f63fff700 1 -- 172.21.15.92:0/1547791370 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5f7c062e80 con 0x7f5f640a3c40 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f640a3c40 msgr2=0x7f5f640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f640a3c40 0x7f5f640a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5f70005fd0 tx=0x7f5f70005f60 comp rx=0 tx=0).stop 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c102aa0 msgr2=0x7f5f7c198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c102aa0 0x7f5f7c198b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00c8f0 tx=0x7f5f6c00ccb0 comp rx=0 tx=0).stop 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 shutdown_connections 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f640a3c40 0x7f5f640a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f7c104640 0x7f5f7c1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f7c102aa0 0x7f5f7c198b80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 --2- 172.21.15.92:0/1547791370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c103ca0 0x7f5f7c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 >> 172.21.15.92:0/1547791370 conn(0x7f5f7c0fe040 msgr2=0x7f5f7c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 shutdown_connections 2024-04-25T20:52:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:52:54.073+0000 7f5f8135c700 1 -- 172.21.15.92:0/1547791370 wait complete. 2024-04-25T20:52:54.681 DEBUG:teuthology.orchestra.run.smithi092:osd.4> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T20:52:54.683 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2024-04-25T20:52:54.683 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T20:52:54.928 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:52:55.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:55 smithi092 ceph-mon[32524]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:55.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:55.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:55.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:55.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:55 smithi092 ceph-mon[32524]: from='osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:52:55.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:55 smithi092 ceph-mon[32524]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:52:55.360 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:55 smithi186 ceph-mon[32529]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:55.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:55.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:55.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:55.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:55 smithi186 ceph-mon[32529]: from='osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:52:55.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:55 smithi186 ceph-mon[32529]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:52:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:55 smithi047 ceph-mon[29195]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:55.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:55.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:55 smithi047 ceph-mon[29195]: from='osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:52:55.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:55 smithi047 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:52:56.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:56 smithi186 ceph-mon[32529]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T20:52:56.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:56 smithi186 ceph-mon[32529]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T20:52:56.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:56 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:56.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:56 smithi186 ceph-mon[32529]: from='osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:52:56.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:56 smithi186 ceph-mon[32529]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:52:56.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:56 smithi092 ceph-mon[32524]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T20:52:56.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:56 smithi092 ceph-mon[32524]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T20:52:56.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:56 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:56.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:56 smithi092 ceph-mon[32524]: from='osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:52:56.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:56 smithi092 ceph-mon[32524]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:52:56.283 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 20:52:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4[43630]: 2024-04-25T20:52:56.039+0000 7f8efc061700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T20:52:56.283 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 20:52:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4[43630]: 2024-04-25T20:52:56.045+0000 7f8ef7e54700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:52:56.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:56 smithi047 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T20:52:56.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:56 smithi047 ceph-mon[29195]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T20:52:56.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:56 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:56.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:56 smithi047 ceph-mon[29195]: from='osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:52:56.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:56 smithi047 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:52:56.826 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T20:52:56.826 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T20:52:56.827 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-25T20:52:56.827 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-25T20:52:56.827 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0166168 s, 631 MB/s 2024-04-25T20:52:56.827 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-25T20:52:57.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:57 smithi092 ceph-mon[32524]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:57.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:57 smithi092 ceph-mon[32524]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:52:57.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:57 smithi092 ceph-mon[32524]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T20:52:57.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:57 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:57.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:57 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:57.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:57 smithi047 ceph-mon[29195]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:57.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:57 smithi047 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:52:57.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:57 smithi047 ceph-mon[29195]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T20:52:57.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:57.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:57.379 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:57 smithi186 ceph-mon[32529]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:57.379 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:57 smithi186 ceph-mon[32529]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:52:57.379 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:57 smithi186 ceph-mon[32529]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T20:52:57.379 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:57 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:57.379 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:57 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:57.407 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2024-04-25T20:52:57.658 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] boot 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi092 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:58.336 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:58.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi092 to 8544M 2024-04-25T20:52:58.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:58.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:58.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:52:58.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:52:58.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] boot 2024-04-25T20:52:58.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T20:52:58.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:58.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi092 2024-04-25T20:52:58.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:58.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:58.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi092 to 8544M 2024-04-25T20:52:58.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:58.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:58.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.507 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:52:58.507 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:52:58.507 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: osd.4 [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] boot 2024-04-25T20:52:58.507 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T20:52:58.507 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:52:58.507 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi092 2024-04-25T20:52:58.508 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.508 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.508 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:58.508 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:58.508 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi092 to 8544M 2024-04-25T20:52:58.508 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:58.509 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:58.509 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:58.509 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:52:59.136 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 -- 172.21.15.186:0/3397877285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9978103570 msgr2=0x7f9978109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:59.136 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 --2- 172.21.15.186:0/3397877285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9978103570 0x7f9978109e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f997000b0a0 tx=0x7f997000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:52:59.136 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 -- 172.21.15.186:0/3397877285 shutdown_connections 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 --2- 172.21.15.186:0/3397877285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9978103570 0x7f9978109e20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 --2- 172.21.15.186:0/3397877285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99781019d0 0x7f9978101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 --2- 172.21.15.186:0/3397877285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9978102bd0 0x7f9978103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 -- 172.21.15.186:0/3397877285 >> 172.21.15.186:0/3397877285 conn(0x7f99780fcf90 msgr2=0x7f99780ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 -- 172.21.15.186:0/3397877285 shutdown_connections 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.133+0000 7f997d507700 1 -- 172.21.15.186:0/3397877285 wait complete. 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f997d507700 1 Processor -- start 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f997d507700 1 -- start start 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f997d507700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99781019d0 0x7f997806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f997d507700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978102bd0 0x7f997819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f997d507700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9978103570 0x7f997819fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:59.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f997d507700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f997810d450 con 0x7f9978103570 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f997d507700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f997810d2d0 con 0x7f9978102bd0 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f997d507700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f997810d5d0 con 0x7f99781019d0 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99781019d0 0x7f997806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99781019d0 0x7f997806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:41710/0 (socket says 172.21.15.186:41710) 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.134+0000 7f9976ffd700 1 -- 172.21.15.186:0/907038019 learned_addr learned my addr 172.21.15.186:0/907038019 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f99777fe700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9978103570 0x7f997819fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f99767fc700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978102bd0 0x7f997819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f9976ffd700 1 -- 172.21.15.186:0/907038019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978102bd0 msgr2=0x7f997819d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f9976ffd700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978102bd0 0x7f997819d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f9976ffd700 1 -- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9978103570 msgr2=0x7f997819fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f9976ffd700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9978103570 0x7f997819fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f9976ffd700 1 -- 172.21.15.186:0/907038019 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99781a0760 con 0x7f99781019d0 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f99767fc700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978102bd0 0x7f997819d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:52:59.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f9976ffd700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99781019d0 0x7f997806bc90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c970 tx=0x7f996c00cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:59.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f995ffff700 1 -- 172.21.15.186:0/907038019 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c004220 con 0x7f99781019d0 2024-04-25T20:52:59.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99781a0a50 con 0x7f99781019d0 2024-04-25T20:52:59.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f995ffff700 1 -- 172.21.15.186:0/907038019 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f996c007d50 con 0x7f99781019d0 2024-04-25T20:52:59.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f995ffff700 1 -- 172.21.15.186:0/907038019 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c003cc0 con 0x7f99781019d0 2024-04-25T20:52:59.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.135+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99781a0fe0 con 0x7f99781019d0 2024-04-25T20:52:59.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.136+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9978069730 con 0x7f99781019d0 2024-04-25T20:52:59.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.137+0000 7f995ffff700 1 -- 172.21.15.186:0/907038019 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f996c0043c0 con 0x7f99781019d0 2024-04-25T20:52:59.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.138+0000 7f995ffff700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99600a3cc0 0x7f99600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:59.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.138+0000 7f995ffff700 1 -- 172.21.15.186:0/907038019 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f996c0cef30 con 0x7f99781019d0 2024-04-25T20:52:59.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.138+0000 7f99767fc700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99600a3cc0 0x7f99600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:59.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.138+0000 7f99767fc700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99600a3cc0 0x7f99600a6180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9968005fd0 tx=0x7f9968005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:59.143 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.142+0000 7f995ffff700 1 -- 172.21.15.186:0/907038019 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f996c014030 con 0x7f99781019d0 2024-04-25T20:52:59.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:59 smithi186 ceph-mon[32529]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:59.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:52:59 smithi186 ceph-mon[32529]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T20:52:59.353 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:52:59.351+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9978062e80 con 0x7f99600a3cc0 2024-04-25T20:52:59.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:59 smithi047 ceph-mon[29195]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:59.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:52:59 smithi047 ceph-mon[29195]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T20:52:59.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:59 smithi092 ceph-mon[32524]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T20:52:59.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:52:59 smithi092 ceph-mon[32524]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T20:53:00.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:00 smithi186 ceph-mon[32529]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:00.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:00.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:00.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:00.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:00 smithi047 ceph-mon[29195]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:00.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:00.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:00.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:00.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:00 smithi092 ceph-mon[32524]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:00.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:00.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:00.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:01.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:01 smithi186 ceph-mon[32529]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:01.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:01 smithi047 ceph-mon[29195]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:01.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:01 smithi092 ceph-mon[32524]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:02 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/521590680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]: dispatch 2024-04-25T20:53:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:02 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]: dispatch 2024-04-25T20:53:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:02 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]': finished 2024-04-25T20:53:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:02 smithi047 ceph-mon[29195]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T20:53:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:02 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:02 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/521590680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]: dispatch 2024-04-25T20:53:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:02 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]: dispatch 2024-04-25T20:53:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:02 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]': finished 2024-04-25T20:53:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:02 smithi186 ceph-mon[32529]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T20:53:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:02 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:02 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/521590680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]: dispatch 2024-04-25T20:53:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:02 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]: dispatch 2024-04-25T20:53:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:02 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21521287-3d9f-4f8d-aa39-4a2d79d2cf34"}]': finished 2024-04-25T20:53:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:02 smithi092 ceph-mon[32524]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T20:53:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:02 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:03.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:03 smithi047 ceph-mon[29195]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:03.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:03 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/650214122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:03.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:03 smithi186 ceph-mon[32529]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:03.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:03 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/650214122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:03.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:03 smithi092 ceph-mon[32524]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:03.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:03 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/650214122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:05.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:05 smithi047 ceph-mon[29195]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:05.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:05 smithi186 ceph-mon[32529]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:05.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:05 smithi092 ceph-mon[32524]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:07.322 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:07 smithi186 ceph-mon[32529]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:07.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:07 smithi047 ceph-mon[29195]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:07.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:07 smithi092 ceph-mon[32524]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:09.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:09 smithi047 ceph-mon[29195]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:09.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:09 smithi186 ceph-mon[32529]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:09.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:09 smithi092 ceph-mon[32524]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:10.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T20:53:10.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:10.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:10 smithi047 ceph-mon[29195]: Deploying daemon osd.5 on smithi186 2024-04-25T20:53:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T20:53:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:10.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:10 smithi186 ceph-mon[32529]: Deploying daemon osd.5 on smithi186 2024-04-25T20:53:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T20:53:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:10 smithi092 ceph-mon[32524]: Deploying daemon osd.5 on smithi186 2024-04-25T20:53:11.359 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:11 smithi186 ceph-mon[32529]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:11.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:11 smithi047 ceph-mon[29195]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:11.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:11 smithi092 ceph-mon[32524]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:13.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:13 smithi186 ceph-mon[32529]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:13.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:13 smithi047 ceph-mon[29195]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:13.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:13 smithi092 ceph-mon[32524]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:15 smithi047 ceph-mon[29195]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:15 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:15 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:15 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:15 smithi186 ceph-mon[32529]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:15.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:15 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:15.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:15 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:15.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:15 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:16.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:15 smithi092 ceph-mon[32524]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:16.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:15 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:16.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:15 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:16.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:15 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:17.859 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:17 smithi186 ceph-mon[32529]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:17.860 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:17.860 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:17.860 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:17.860 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:17.860 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:17 smithi047 ceph-mon[29195]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:17 smithi092 ceph-mon[32524]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:18.221 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2024-04-25T20:53:18.222 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.220+0000 7f995ffff700 1 -- 172.21.15.186:0/907038019 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9978062e80 con 0x7f99600a3cc0 2024-04-25T20:53:18.223 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99600a3cc0 msgr2=0x7f99600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:18.223 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99600a3cc0 0x7f99600a6180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9968005fd0 tx=0x7f9968005dc0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99781019d0 msgr2=0x7f997806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99781019d0 0x7f997806bc90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c970 tx=0x7f996c00cd30 comp rx=0 tx=0).stop 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 shutdown_connections 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99600a3cc0 0x7f99600a6180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9978103570 0x7f997819fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99781019d0 0x7f997806bc90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 --2- 172.21.15.186:0/907038019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978102bd0 0x7f997819d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 >> 172.21.15.186:0/907038019 conn(0x7f99780fcf90 msgr2=0x7f9978105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 shutdown_connections 2024-04-25T20:53:18.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:53:18.222+0000 7f997d507700 1 -- 172.21.15.186:0/907038019 wait complete. 2024-04-25T20:53:18.840 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T20:53:18.842 INFO:tasks.cephadm:Deploying osd.6 on smithi047 with /dev/vg_nvme/lv_2... 2024-04-25T20:53:18.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T20:53:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:19.443 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:19 smithi047 ceph-mon[29195]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:19.443 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:19.443 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:19.443 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:19.444 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:19 smithi047 ceph-mon[29195]: from='osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:53:19.444 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:19 smithi047 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:53:19.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:19 smithi186 ceph-mon[32529]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:19.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:19.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:19.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:19.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:19 smithi186 ceph-mon[32529]: from='osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:53:19.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:19 smithi186 ceph-mon[32529]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:53:19.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:19 smithi092 ceph-mon[32524]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:19.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:19.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:19.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:19.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:19 smithi092 ceph-mon[32524]: from='osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:53:19.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:19 smithi092 ceph-mon[32524]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:53:20.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:20 smithi047 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T20:53:20.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:20 smithi047 ceph-mon[29195]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T20:53:20.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:20.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:20 smithi047 ceph-mon[29195]: from='osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:53:20.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:20 smithi047 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:53:20.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:20 smithi186 ceph-mon[32529]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T20:53:20.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:20 smithi186 ceph-mon[32529]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T20:53:20.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:20.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:20 smithi186 ceph-mon[32529]: from='osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:53:20.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:20 smithi186 ceph-mon[32529]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:53:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:20 smithi092 ceph-mon[32524]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T20:53:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:20 smithi092 ceph-mon[32524]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T20:53:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:20 smithi092 ceph-mon[32524]: from='osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:53:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:20 smithi092 ceph-mon[32524]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:53:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T20:53:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T20:53:21.132 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-04-25T20:53:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-04-25T20:53:21.132 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00890938 s, 1.2 GB/s 2024-04-25T20:53:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-04-25T20:53:21.290 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:21 smithi186 ceph-mon[32529]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:21.291 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:21 smithi186 ceph-mon[32529]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:53:21.291 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:21 smithi186 ceph-mon[32529]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T20:53:21.291 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:21 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:21.291 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:21 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:21.291 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:21 smithi186 ceph-mon[32529]: from='osd.5 ' entity='osd.5' 2024-04-25T20:53:21.291 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 20:53:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T20:53:21.089+0000 7feaf5e91700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T20:53:21.291 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 20:53:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T20:53:21.289+0000 7feaedc7c700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:53:21.441 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:21 smithi047 ceph-mon[29195]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:21.441 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:21 smithi047 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:53:21.442 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:21 smithi047 ceph-mon[29195]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T20:53:21.442 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:21 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:21.442 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:21 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:21.442 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:21 smithi047 ceph-mon[29195]: from='osd.5 ' entity='osd.5' 2024-04-25T20:53:21.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:21 smithi092 ceph-mon[32524]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:21.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:21 smithi092 ceph-mon[32524]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:53:21.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:21 smithi092 ceph-mon[32524]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T20:53:21.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:21 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:21.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:21 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:21.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:21 smithi092 ceph-mon[32524]: from='osd.5 ' entity='osd.5' 2024-04-25T20:53:21.761 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-04-25T20:53:22.010 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:22.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:53:22.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:53:22.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T20:53:22.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:22.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi186 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:22.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi186 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:22.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:53:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:53:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T20:53:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi186 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:22.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:23.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:23 smithi047 ceph-mon[29195]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:23.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:23 smithi047 ceph-mon[29195]: osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] boot 2024-04-25T20:53:23.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:23 smithi047 ceph-mon[29195]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T20:53:23.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 -- 172.21.15.47:0/611454797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 msgr2=0x7f5154109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 --2- 172.21.15.47:0/611454797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 0x7f5154109df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f514c00b0d0 tx=0x7f514c00b3e0 comp rx=0 tx=0).stop 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 -- 172.21.15.47:0/611454797 shutdown_connections 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 --2- 172.21.15.47:0/611454797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 0x7f5154109df0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 --2- 172.21.15.47:0/611454797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51541019a0 0x7f5154101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 --2- 172.21.15.47:0/611454797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154102ba0 0x7f5154103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 -- 172.21.15.47:0/611454797 >> 172.21.15.47:0/611454797 conn(0x7f51540fcf40 msgr2=0x7f51540ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 -- 172.21.15.47:0/611454797 shutdown_connections 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 -- 172.21.15.47:0/611454797 wait complete. 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.444+0000 7f515903a700 1 Processor -- start 2024-04-25T20:53:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515903a700 1 -- start start 2024-04-25T20:53:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515903a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51541019a0 0x7f51541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515903a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5154102ba0 0x7f5154198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515903a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 0x7f515419ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515903a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d420 con 0x7f5154103540 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515903a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d2a0 con 0x7f51541019a0 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515903a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d5a0 con 0x7f5154102ba0 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 0x7f515419ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 0x7f515419ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36030/0 (socket says 172.21.15.47:36030) 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515359e700 1 -- 172.21.15.47:0/727632788 learned_addr learned my addr 172.21.15.47:0/727632788 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f5152d9d700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51541019a0 0x7f51541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.445+0000 7f515259c700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5154102ba0 0x7f5154198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f5152d9d700 1 -- 172.21.15.47:0/727632788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5154102ba0 msgr2=0x7f5154198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f5152d9d700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5154102ba0 0x7f5154198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f5152d9d700 1 -- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 msgr2=0x7f515419ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f5152d9d700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 0x7f515419ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f5152d9d700 1 -- 172.21.15.47:0/727632788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51541a05d0 con 0x7f51541019a0 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f515359e700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 0x7f515419ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f515259c700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5154102ba0 0x7f5154198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f5152d9d700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51541019a0 0x7f51541989c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f514400d740 tx=0x7f514400da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f513bfff700 1 -- 172.21.15.47:0/727632788 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144017070 con 0x7f51541019a0 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f513bfff700 1 -- 172.21.15.47:0/727632788 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f514400f690 con 0x7f51541019a0 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51541a08c0 con 0x7f51541019a0 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f513bfff700 1 -- 172.21.15.47:0/727632788 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144003b40 con 0x7f51541019a0 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.446+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51541a0e00 con 0x7f51541019a0 2024-04-25T20:53:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.447+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5154069730 con 0x7f51541019a0 2024-04-25T20:53:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.448+0000 7f513bfff700 1 -- 172.21.15.47:0/727632788 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f514400f840 con 0x7f51541019a0 2024-04-25T20:53:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.449+0000 7f513bfff700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f513c0a3c40 0x7f513c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.449+0000 7f515259c700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f513c0a3c40 0x7f513c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.449+0000 7f513bfff700 1 -- 172.21.15.47:0/727632788 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f51440cf170 con 0x7f51541019a0 2024-04-25T20:53:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.449+0000 7f515259c700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f513c0a3c40 0x7f513c0a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5148005950 tx=0x7f5148005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.453+0000 7f513bfff700 1 -- 172.21.15.47:0/727632788 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f514409c950 con 0x7f51541019a0 2024-04-25T20:53:23.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:23 smithi186 ceph-mon[32529]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:23.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:23 smithi186 ceph-mon[32529]: osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] boot 2024-04-25T20:53:23.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:23 smithi186 ceph-mon[32529]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T20:53:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:23.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:23 smithi092 ceph-mon[32524]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:53:23.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:23 smithi092 ceph-mon[32524]: osd.5 [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] boot 2024-04-25T20:53:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:23 smithi092 ceph-mon[32524]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T20:53:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:53:23.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:23.670+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f51540feef0 con 0x7f513c0a3c40 2024-04-25T20:53:24.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:24 smithi186 ceph-mon[32529]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T20:53:24.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:24 smithi186 ceph-mon[32529]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:24.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:24.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:24.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:24.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:24 smithi047 ceph-mon[29195]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T20:53:24.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:24 smithi047 ceph-mon[29195]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:24.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:24.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:24.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:24.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:24 smithi092 ceph-mon[32524]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T20:53:24.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:24 smithi092 ceph-mon[32524]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:24.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:24.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:24.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:25.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:25 smithi047 ceph-mon[29195]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:25.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:25 smithi047 ceph-mon[29195]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T20:53:25.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:25 smithi186 ceph-mon[32529]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:25.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:25 smithi186 ceph-mon[32529]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T20:53:25.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:25 smithi092 ceph-mon[32524]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:25.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:25 smithi092 ceph-mon[32524]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T20:53:26.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:26 smithi186 ceph-mon[32529]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T20:53:26.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:26 smithi092 ceph-mon[32524]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T20:53:26.579 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:26 smithi047 ceph-mon[29195]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T20:53:27.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:27 smithi186 ceph-mon[32529]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:27.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:27 smithi186 ceph-mon[32529]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T20:53:27.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:27 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2757072313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7726429-12e9-4dd8-9bae-8aa979f0c447"}]: dispatch 2024-04-25T20:53:27.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:27 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2757072313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7726429-12e9-4dd8-9bae-8aa979f0c447"}]': finished 2024-04-25T20:53:27.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:27 smithi186 ceph-mon[32529]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T20:53:27.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:27 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:27.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:27 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/340000835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:27.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:27 smithi092 ceph-mon[32524]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:27.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:27 smithi092 ceph-mon[32524]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T20:53:27.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:27 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2757072313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7726429-12e9-4dd8-9bae-8aa979f0c447"}]: dispatch 2024-04-25T20:53:27.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:27 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2757072313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7726429-12e9-4dd8-9bae-8aa979f0c447"}]': finished 2024-04-25T20:53:27.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:27 smithi092 ceph-mon[32524]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T20:53:27.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:27 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:27.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:27 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/340000835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:27 smithi047 ceph-mon[29195]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:27 smithi047 ceph-mon[29195]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T20:53:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:27 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2757072313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7726429-12e9-4dd8-9bae-8aa979f0c447"}]: dispatch 2024-04-25T20:53:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:27 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2757072313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7726429-12e9-4dd8-9bae-8aa979f0c447"}]': finished 2024-04-25T20:53:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:27 smithi047 ceph-mon[29195]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T20:53:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:27 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:27 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/340000835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:29.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:29 smithi186 ceph-mon[32529]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:29.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:29 smithi092 ceph-mon[32524]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:29.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:29 smithi047 ceph-mon[29195]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:31.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:31 smithi047 ceph-mon[29195]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T20:53:31.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:31 smithi186 ceph-mon[32529]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T20:53:31.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:31 smithi092 ceph-mon[32524]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T20:53:33.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:33 smithi047 ceph-mon[29195]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:33.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T20:53:33.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:33.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:33 smithi186 ceph-mon[32529]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:33.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T20:53:33.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:33.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:33 smithi092 ceph-mon[32524]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:33.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T20:53:33.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:34.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:34 smithi047 ceph-mon[29195]: Deploying daemon osd.6 on smithi047 2024-04-25T20:53:34.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:34 smithi092 ceph-mon[32524]: Deploying daemon osd.6 on smithi047 2024-04-25T20:53:34.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:34 smithi186 ceph-mon[32529]: Deploying daemon osd.6 on smithi047 2024-04-25T20:53:35.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:35 smithi047 ceph-mon[29195]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:35.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:35 smithi092 ceph-mon[32524]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:35.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:35 smithi186 ceph-mon[32529]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:37.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:37 smithi092 ceph-mon[32524]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:37.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:37 smithi047 ceph-mon[29195]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:37.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:37 smithi186 ceph-mon[32529]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:39.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:39 smithi047 ceph-mon[29195]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:39.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:39.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:39.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:39.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:39 smithi092 ceph-mon[32524]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:39.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:39.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:39.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:39.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:39 smithi186 ceph-mon[32529]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:39.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:39.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:39.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:41.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:41 smithi092 ceph-mon[32524]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:41.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:41.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:41.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:41 smithi047 ceph-mon[29195]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:41.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:41.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:41.583 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:41 smithi186 ceph-mon[32529]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:41.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:41.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:41.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:41.876 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 6 on host 'smithi047' 2024-04-25T20:53:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.874+0000 7f513bfff700 1 -- 172.21.15.47:0/727632788 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f51540feef0 con 0x7f513c0a3c40 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f513c0a3c40 msgr2=0x7f513c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f513c0a3c40 0x7f513c0a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5148005950 tx=0x7f5148005f50 comp rx=0 tx=0).stop 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51541019a0 msgr2=0x7f51541989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51541019a0 0x7f51541989c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f514400d740 tx=0x7f514400da50 comp rx=0 tx=0).stop 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 shutdown_connections 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f513c0a3c40 0x7f513c0a6100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5154103540 0x7f515419ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51541019a0 0x7f51541989c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 --2- 172.21.15.47:0/727632788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5154102ba0 0x7f5154198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 >> 172.21.15.47:0/727632788 conn(0x7f51540fcf40 msgr2=0x7f51540fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 shutdown_connections 2024-04-25T20:53:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:53:41.877+0000 7f515903a700 1 -- 172.21.15.47:0/727632788 wait complete. 2024-04-25T20:53:42.440 DEBUG:teuthology.orchestra.run.smithi047:osd.6> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T20:53:42.442 INFO:tasks.cephadm:Deploying osd.7 on smithi092 with /dev/vg_nvme/lv_2... 2024-04-25T20:53:42.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T20:53:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:53:43.100 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:42 smithi092 ceph-mon[32524]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:43.100 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:42 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:43.100 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:42 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:43.100 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:42 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:43.100 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:42 smithi092 ceph-mon[32524]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T20:53:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:42 smithi047 ceph-mon[29195]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:42 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:42 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:42 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:42 smithi047 ceph-mon[29195]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T20:53:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:42 smithi186 ceph-mon[32529]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:42 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:42 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:43.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:42 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:43.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:42 smithi186 ceph-mon[32529]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T20:53:43.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:43 smithi047 ceph-mon[29195]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T20:53:43.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:43 smithi047 ceph-mon[29195]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T20:53:43.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:43 smithi047 ceph-mon[29195]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:53:43.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:44.146 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:43 smithi092 ceph-mon[32524]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T20:53:44.146 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:43 smithi092 ceph-mon[32524]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T20:53:44.146 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:43 smithi092 ceph-mon[32524]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:53:44.146 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:44.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:43 smithi186 ceph-mon[32529]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T20:53:44.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:43 smithi186 ceph-mon[32529]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T20:53:44.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:43 smithi186 ceph-mon[32529]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:53:44.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T20:53:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T20:53:44.795 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-04-25T20:53:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-04-25T20:53:44.796 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117738 s, 891 MB/s 2024-04-25T20:53:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-04-25T20:53:45.075 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:44 smithi047 ceph-mon[29195]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:45.075 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:44 smithi047 ceph-mon[29195]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:53:45.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:44 smithi047 ceph-mon[29195]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T20:53:45.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:44 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:45.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:44 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:45.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:44 smithi047 ceph-mon[29195]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' 2024-04-25T20:53:45.076 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 20:53:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T20:53:44.681+0000 7f14251db700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T20:53:45.076 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 20:53:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T20:53:44.693+0000 7f141c7c5700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:53:45.109 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:44 smithi092 ceph-mon[32524]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:45.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:44 smithi092 ceph-mon[32524]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:53:45.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:44 smithi092 ceph-mon[32524]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T20:53:45.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:44 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:45.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:44 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:45.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:44 smithi092 ceph-mon[32524]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' 2024-04-25T20:53:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:44 smithi186 ceph-mon[32529]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:44 smithi186 ceph-mon[32529]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:53:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:44 smithi186 ceph-mon[32529]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T20:53:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:44 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:44 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:44 smithi186 ceph-mon[32529]: from='osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922]' entity='osd.6' 2024-04-25T20:53:45.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-04-25T20:53:45.634 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:53:45.964 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi047 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi047 to 5697M 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:45.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:45.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] boot 2024-04-25T20:53:45.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T20:53:45.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi047 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi047 to 5697M 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:46.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:46.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:46.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] boot 2024-04-25T20:53:46.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T20:53:46.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:53:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:53:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi047 2024-04-25T20:53:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:46.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi047 to 5697M 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: osd.6 [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] boot 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T20:53:46.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:53:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 -- 172.21.15.92:0/4047741662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398103d30 msgr2=0x7fc398104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 --2- 172.21.15.92:0/4047741662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398103d30 0x7fc398104190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc380009990 tx=0x7fc380009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 -- 172.21.15.92:0/4047741662 shutdown_connections 2024-04-25T20:53:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 --2- 172.21.15.92:0/4047741662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3981046d0 0x7fc39810af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 --2- 172.21.15.92:0/4047741662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc398100d50 0x7fc398101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 --2- 172.21.15.92:0/4047741662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398103d30 0x7fc398104190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 -- 172.21.15.92:0/4047741662 >> 172.21.15.92:0/4047741662 conn(0x7fc3980fc6d0 msgr2=0x7fc3980feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 -- 172.21.15.92:0/4047741662 shutdown_connections 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.106+0000 7fc39d853700 1 -- 172.21.15.92:0/4047741662 wait complete. 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc39d853700 1 Processor -- start 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc39d853700 1 -- start start 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc39d853700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398100d50 0x7fc398198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc39d853700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc398103d30 0x7fc3981990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc39d853700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3981046d0 0x7fc3981a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc39d853700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810e510 con 0x7fc3981046d0 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc39d853700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810e390 con 0x7fc398100d50 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc39d853700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810e690 con 0x7fc398103d30 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398100d50 0x7fc398198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398100d50 0x7fc398198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55654/0 (socket says 172.21.15.92:55654) 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 -- 172.21.15.92:0/882515540 learned_addr learned my addr 172.21.15.92:0/882515540 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-25T20:53:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc3977fe700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3981046d0 0x7fc3981a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 -- 172.21.15.92:0/882515540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc398103d30 msgr2=0x7fc3981990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc398103d30 0x7fc3981990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 -- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3981046d0 msgr2=0x7fc3981a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3981046d0 0x7fc3981a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 -- 172.21.15.92:0/882515540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3981a0810 con 0x7fc398100d50 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc3977fe700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3981046d0 0x7fc3981a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.107+0000 7fc396ffd700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398100d50 0x7fc398198bb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc38800ada0 tx=0x7fc38800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.108+0000 7fc39c851700 1 -- 172.21.15.92:0/882515540 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388017070 con 0x7fc398100d50 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.108+0000 7fc39c851700 1 -- 172.21.15.92:0/882515540 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc388004bb0 con 0x7fc398100d50 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.108+0000 7fc39c851700 1 -- 172.21.15.92:0/882515540 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388004020 con 0x7fc398100d50 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.108+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3981a0b00 con 0x7fc398100d50 2024-04-25T20:53:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.108+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3981a0ff0 con 0x7fc398100d50 2024-04-25T20:53:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.109+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc398069730 con 0x7fc398100d50 2024-04-25T20:53:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.110+0000 7fc39c851700 1 -- 172.21.15.92:0/882515540 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc38801a070 con 0x7fc398100d50 2024-04-25T20:53:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.110+0000 7fc39c851700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc3840a3c40 0x7fc3840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.110+0000 7fc39c851700 1 -- 172.21.15.92:0/882515540 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fc3880ce7a0 con 0x7fc398100d50 2024-04-25T20:53:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.110+0000 7fc3967fc700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc3840a3c40 0x7fc3840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.111+0000 7fc3967fc700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc3840a3c40 0x7fc3840a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc38000b5c0 tx=0x7fc380003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:47.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.115+0000 7fc39c851700 1 -- 172.21.15.92:0/882515540 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc38809be90 con 0x7fc398100d50 2024-04-25T20:53:47.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:53:47.321+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc398062e80 con 0x7fc3840a3c40 2024-04-25T20:53:47.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:47 smithi186 ceph-mon[32529]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:47.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:47 smithi186 ceph-mon[32529]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T20:53:47.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:47.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:47.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:48.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:47 smithi092 ceph-mon[32524]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:48.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:47 smithi092 ceph-mon[32524]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T20:53:48.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:48.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:48.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:48.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:47 smithi047 ceph-mon[29195]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:53:48.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:47 smithi047 ceph-mon[29195]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T20:53:48.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:53:48.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:53:48.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:48 smithi092 ceph-mon[32524]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:48.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:48 smithi186 ceph-mon[32529]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:49.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:48 smithi047 ceph-mon[29195]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:49.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:49 smithi186 ceph-mon[32529]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:50.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:49 smithi092 ceph-mon[32524]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:50.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:49 smithi047 ceph-mon[29195]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:50.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:50 smithi186 ceph-mon[32529]: from='client.? 172.21.15.92:0/1047992929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]: dispatch 2024-04-25T20:53:50.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:50 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]: dispatch 2024-04-25T20:53:50.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:50 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]': finished 2024-04-25T20:53:50.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:50 smithi186 ceph-mon[32529]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T20:53:50.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:53:50.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:50 smithi186 ceph-mon[32529]: from='client.? 172.21.15.92:0/2505423719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:51.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:50 smithi092 ceph-mon[32524]: from='client.? 172.21.15.92:0/1047992929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]: dispatch 2024-04-25T20:53:51.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:50 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]: dispatch 2024-04-25T20:53:51.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:50 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]': finished 2024-04-25T20:53:51.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:50 smithi092 ceph-mon[32524]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T20:53:51.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:53:51.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:50 smithi092 ceph-mon[32524]: from='client.? 172.21.15.92:0/2505423719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:50 smithi047 ceph-mon[29195]: from='client.? 172.21.15.92:0/1047992929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]: dispatch 2024-04-25T20:53:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:50 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]: dispatch 2024-04-25T20:53:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:50 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f86109-f4e5-4770-88b9-6454fa1927bd"}]': finished 2024-04-25T20:53:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:50 smithi047 ceph-mon[29195]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T20:53:51.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:53:51.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:50 smithi047 ceph-mon[29195]: from='client.? 172.21.15.92:0/2505423719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:51.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:51 smithi186 ceph-mon[32529]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:52.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:51 smithi092 ceph-mon[32524]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:52.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:51 smithi047 ceph-mon[29195]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:52.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:52 smithi186 ceph-mon[32529]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:53.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:52 smithi092 ceph-mon[32524]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:53.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:52 smithi047 ceph-mon[29195]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:55.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:54 smithi092 ceph-mon[32524]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:55.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:54 smithi186 ceph-mon[32529]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:54 smithi047 ceph-mon[29195]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:57.216 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:56 smithi092 ceph-mon[32524]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:57.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:56 smithi186 ceph-mon[32529]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:57.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:56 smithi047 ceph-mon[29195]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:58.178 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:57 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T20:53:58.179 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:57 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:58.179 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:57 smithi092 ceph-mon[32524]: Deploying daemon osd.7 on smithi092 2024-04-25T20:53:58.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:57 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T20:53:58.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:57 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:58.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:57 smithi186 ceph-mon[32529]: Deploying daemon osd.7 on smithi092 2024-04-25T20:53:58.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T20:53:58.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:58.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:57 smithi047 ceph-mon[29195]: Deploying daemon osd.7 on smithi092 2024-04-25T20:53:59.193 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:53:58 smithi092 ceph-mon[32524]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:59.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:53:58 smithi186 ceph-mon[32529]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:53:59.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:53:58 smithi047 ceph-mon[29195]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:01.144 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:00 smithi092 ceph-mon[32524]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:01.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:00 smithi186 ceph-mon[32529]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:01.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:00 smithi047 ceph-mon[29195]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:03.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:03 smithi092 ceph-mon[32524]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:03.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:03 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:03.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:03 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:03.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:03 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:03.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:03 smithi186 ceph-mon[32529]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:03.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:03 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:03.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:03 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:03.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:03 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:04.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:03 smithi047 ceph-mon[29195]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:04.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:03 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:04.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:03 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:04.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:03 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:05 smithi186 ceph-mon[32529]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:05 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:05 smithi092 ceph-mon[32524]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:05 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:05 smithi047 ceph-mon[29195]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:05 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:06.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.493+0000 7fc39c851700 1 -- 172.21.15.92:0/882515540 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc398062e80 con 0x7fc3840a3c40 2024-04-25T20:54:06.495 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 7 on host 'smithi092' 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc3840a3c40 msgr2=0x7fc3840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc3840a3c40 0x7fc3840a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc38000b5c0 tx=0x7fc380003860 comp rx=0 tx=0).stop 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398100d50 msgr2=0x7fc398198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398100d50 0x7fc398198bb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc38800ada0 tx=0x7fc38800c330 comp rx=0 tx=0).stop 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 shutdown_connections 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc3840a3c40 0x7fc3840a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3981046d0 0x7fc3981a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc398100d50 0x7fc398198bb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 --2- 172.21.15.92:0/882515540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc398103d30 0x7fc3981990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:06.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 >> 172.21.15.92:0/882515540 conn(0x7fc3980fc6d0 msgr2=0x7fc398107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:06.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 shutdown_connections 2024-04-25T20:54:06.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:54:06.496+0000 7fc39d853700 1 -- 172.21.15.92:0/882515540 wait complete. 2024-04-25T20:54:07.073 DEBUG:teuthology.orchestra.run.smithi092:osd.7> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T20:54:07.075 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2024-04-25T20:54:07.075 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T20:54:07.322 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:54:07.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:07 smithi186 ceph-mon[32529]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:07.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:07.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:07.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:07.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:07 smithi186 ceph-mon[32529]: from='osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:54:07.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:07 smithi186 ceph-mon[32529]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:54:07.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:07 smithi092 ceph-mon[32524]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:07.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:07.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:07 smithi092 ceph-mon[32524]: from='osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:54:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:07 smithi092 ceph-mon[32524]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:54:07.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:07 smithi047 ceph-mon[29195]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:07.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:07.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:07.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:07.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:07 smithi047 ceph-mon[29195]: from='osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:54:07.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:07 smithi047 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:54:08.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:08 smithi186 ceph-mon[32529]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T20:54:08.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:08 smithi186 ceph-mon[32529]: from='osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:54:08.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:08 smithi186 ceph-mon[32529]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T20:54:08.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:08 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:08.560 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:08 smithi186 ceph-mon[32529]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:54:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:08 smithi092 ceph-mon[32524]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T20:54:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:08 smithi092 ceph-mon[32524]: from='osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:54:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:08 smithi092 ceph-mon[32524]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T20:54:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:08 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:08.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:08 smithi092 ceph-mon[32524]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:54:08.783 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 20:54:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T20:54:08.464+0000 7ff496bc1700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T20:54:08.783 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 20:54:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T20:54:08.489+0000 7ff4929b4700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:54:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:08 smithi047 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T20:54:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:08 smithi047 ceph-mon[29195]: from='osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:54:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:08 smithi047 ceph-mon[29195]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T20:54:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:08 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:08 smithi047 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:54:09.117 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T20:54:09.117 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T20:54:09.117 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-25T20:54:09.117 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-25T20:54:09.117 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115896 s, 905 MB/s 2024-04-25T20:54:09.117 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-25T20:54:09.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:09 smithi092 ceph-mon[32524]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:09.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:09 smithi092 ceph-mon[32524]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:54:09.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:09 smithi092 ceph-mon[32524]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T20:54:09.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:09 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:09.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:09 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:09.705 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:09 smithi186 ceph-mon[32529]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:09.705 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:09 smithi186 ceph-mon[32529]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:54:09.705 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:09 smithi186 ceph-mon[32529]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T20:54:09.705 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:09 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:09.705 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:09 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:09.732 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2024-04-25T20:54:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:09 smithi047 ceph-mon[29195]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:54:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:09 smithi047 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:54:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:09 smithi047 ceph-mon[29195]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T20:54:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:09 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:09 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:09.977 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] boot 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:10.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:10.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:10 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] boot 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:10.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:10 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:54:10.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:54:10.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: osd.7 [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] boot 2024-04-25T20:54:10.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T20:54:10.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:54:10.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:10.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:10.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:10.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:10.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:10 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 -- 172.21.15.186:0/2125287797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914102f80 msgr2=0x7f89141033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2125287797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914102f80 0x7f89141033a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f89040099c0 tx=0x7f8904009cd0 comp rx=0 tx=0).stop 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 -- 172.21.15.186:0/2125287797 shutdown_connections 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2125287797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8914102660 0x7f8914109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2125287797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914102f80 0x7f89141033a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2125287797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8914101cc0 0x7f8914102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 -- 172.21.15.186:0/2125287797 >> 172.21.15.186:0/2125287797 conn(0x7f89140fc710 msgr2=0x7f89140feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 -- 172.21.15.186:0/2125287797 shutdown_connections 2024-04-25T20:54:11.436 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 -- 172.21.15.186:0/2125287797 wait complete. 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.433+0000 7f8918f6b700 1 Processor -- start 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8918f6b700 1 -- start start 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8918f6b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914101cc0 0x7f8914198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8918f6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8914102660 0x7f89141991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8918f6b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8914102f80 0x7f89141a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8918f6b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f891410d480 con 0x7f8914102f80 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8918f6b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f891410d300 con 0x7f8914102660 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8918f6b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f891410d600 con 0x7f8914101cc0 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914101cc0 0x7f8914198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914101cc0 0x7f8914198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:48370/0 (socket says 172.21.15.186:48370) 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 -- 172.21.15.186:0/2358252153 learned_addr learned my addr 172.21.15.186:0/2358252153 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8912d9d700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8914102f80 0x7f89141a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:11.437 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f8909d9b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8914102660 0x7f89141991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:11.438 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 -- 172.21.15.186:0/2358252153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8914102660 msgr2=0x7f89141991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:11.438 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8914102660 0x7f89141991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:11.438 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 -- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8914102f80 msgr2=0x7f89141a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:11.438 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8914102f80 0x7f89141a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:11.438 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 -- 172.21.15.186:0/2358252153 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89141a0910 con 0x7f8914101cc0 2024-04-25T20:54:11.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.434+0000 7f891259c700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914101cc0 0x7f8914198c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8904006010 tx=0x7f89040047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:11.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.435+0000 7f890bfff700 1 -- 172.21.15.186:0/2358252153 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904027070 con 0x7f8914101cc0 2024-04-25T20:54:11.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.435+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89141a0ba0 con 0x7f8914101cc0 2024-04-25T20:54:11.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.435+0000 7f890bfff700 1 -- 172.21.15.186:0/2358252153 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89040041e0 con 0x7f8914101cc0 2024-04-25T20:54:11.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.435+0000 7f890bfff700 1 -- 172.21.15.186:0/2358252153 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890400f460 con 0x7f8914101cc0 2024-04-25T20:54:11.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.435+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89141a10d0 con 0x7f8914101cc0 2024-04-25T20:54:11.440 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.436+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8914069730 con 0x7f8914101cc0 2024-04-25T20:54:11.440 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.437+0000 7f890bfff700 1 -- 172.21.15.186:0/2358252153 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f890402a070 con 0x7f8914101cc0 2024-04-25T20:54:11.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.437+0000 7f890bfff700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f89000a3c40 0x7f89000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:11.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.437+0000 7f890bfff700 1 -- 172.21.15.186:0/2358252153 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f89040d0c40 con 0x7f8914101cc0 2024-04-25T20:54:11.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.438+0000 7f8909d9b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f89000a3c40 0x7f89000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:11.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.438+0000 7f8909d9b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f89000a3c40 0x7f89000a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005950 tx=0x7f88fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:11.442 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.441+0000 7f890bfff700 1 -- 172.21.15.186:0/2358252153 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8904031070 con 0x7f8914101cc0 2024-04-25T20:54:11.653 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:11.652+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f89140fe8e0 con 0x7f89000a3c40 2024-04-25T20:54:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:11 smithi186 ceph-mon[32529]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:11 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi092 2024-04-25T20:54:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:11 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi092 to 5696M 2024-04-25T20:54:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:11 smithi186 ceph-mon[32529]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T20:54:11.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:11 smithi092 ceph-mon[32524]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:11 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi092 2024-04-25T20:54:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:11 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi092 to 5696M 2024-04-25T20:54:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:11 smithi092 ceph-mon[32524]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T20:54:11.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:11 smithi047 ceph-mon[29195]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:11.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:11 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi092 2024-04-25T20:54:11.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:11 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi092 to 5696M 2024-04-25T20:54:11.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:11 smithi047 ceph-mon[29195]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T20:54:12.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:12 smithi186 ceph-mon[32529]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T20:54:12.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:12 smithi186 ceph-mon[32529]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T20:54:12.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:12 smithi186 ceph-mon[32529]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:12.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:54:12.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:54:12.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:12 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:12 smithi092 ceph-mon[32524]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T20:54:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:12 smithi092 ceph-mon[32524]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T20:54:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:12 smithi092 ceph-mon[32524]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:54:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:54:12.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:12 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:12.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:12 smithi047 ceph-mon[29195]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T20:54:12.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:12 smithi047 ceph-mon[29195]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T20:54:12.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:12 smithi047 ceph-mon[29195]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:12.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:54:12.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:54:12.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:12 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:13.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:13 smithi186 ceph-mon[32529]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:13.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:13 smithi186 ceph-mon[32529]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T20:54:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:13 smithi092 ceph-mon[32524]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:13 smithi092 ceph-mon[32524]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T20:54:14.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:13 smithi047 ceph-mon[29195]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:14.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:13 smithi047 ceph-mon[29195]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T20:54:14.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:14 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/2942097905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]: dispatch 2024-04-25T20:54:14.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:14 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]: dispatch 2024-04-25T20:54:14.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:14 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]': finished 2024-04-25T20:54:14.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:14 smithi186 ceph-mon[32529]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T20:54:14.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:14 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:15.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:14 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/2942097905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]: dispatch 2024-04-25T20:54:15.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:14 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]: dispatch 2024-04-25T20:54:15.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:14 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]': finished 2024-04-25T20:54:15.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:14 smithi092 ceph-mon[32524]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T20:54:15.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:14 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:15.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:14 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/2942097905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]: dispatch 2024-04-25T20:54:15.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:14 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]: dispatch 2024-04-25T20:54:15.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:14 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df80bc74-f3eb-43e1-ac44-7061cdddf622"}]': finished 2024-04-25T20:54:15.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:14 smithi047 ceph-mon[29195]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T20:54:15.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:14 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:15 smithi186 ceph-mon[32529]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:15 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/3718150751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:54:16.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:15 smithi092 ceph-mon[32524]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:16.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:15 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/3718150751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:54:16.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:15 smithi047 ceph-mon[29195]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:16.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:15 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/3718150751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:54:17.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:17 smithi186 ceph-mon[32529]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:17 smithi092 ceph-mon[32524]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:18.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:17 smithi047 ceph-mon[29195]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:19.853 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:19 smithi186 ceph-mon[32529]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:20.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:19 smithi092 ceph-mon[32524]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:20.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:19 smithi047 ceph-mon[29195]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:21.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:21 smithi186 ceph-mon[32529]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:22.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:21 smithi092 ceph-mon[32524]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:22.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:21 smithi047 ceph-mon[29195]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:22.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T20:54:22.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:23.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T20:54:23.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:23.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T20:54:23.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:23.914 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:23 smithi186 ceph-mon[32529]: Deploying daemon osd.8 on smithi186 2024-04-25T20:54:23.914 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:23 smithi186 ceph-mon[32529]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:24.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:23 smithi092 ceph-mon[32524]: Deploying daemon osd.8 on smithi186 2024-04-25T20:54:24.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:23 smithi092 ceph-mon[32524]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:23 smithi047 ceph-mon[29195]: Deploying daemon osd.8 on smithi186 2024-04-25T20:54:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:23 smithi047 ceph-mon[29195]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:25.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:25 smithi186 ceph-mon[32529]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:25 smithi092 ceph-mon[32524]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:25 smithi047 ceph-mon[29195]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:27.709 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:27 smithi186 ceph-mon[32529]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:27.709 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:27 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:27.709 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:27 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:27.709 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:27 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:28.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:27 smithi092 ceph-mon[32524]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:28.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:27 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:28.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:27 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:28.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:27 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:27 smithi047 ceph-mon[29195]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:27 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:27 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:27 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:29 smithi186 ceph-mon[32529]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:29.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:30.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:29 smithi092 ceph-mon[32524]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:30.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:30.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:29 smithi047 ceph-mon[29195]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:30.674 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.673+0000 7f890bfff700 1 -- 172.21.15.186:0/2358252153 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89140fe8e0 con 0x7f89000a3c40 2024-04-25T20:54:30.674 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2024-04-25T20:54:30.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f89000a3c40 msgr2=0x7f89000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:30.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f89000a3c40 0x7f89000a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005950 tx=0x7f88fc0058e0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914101cc0 msgr2=0x7f8914198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:30.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914101cc0 0x7f8914198c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8904006010 tx=0x7f89040047b0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 shutdown_connections 2024-04-25T20:54:30.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f89000a3c40 0x7f89000a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8914102f80 0x7f89141a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8914101cc0 0x7f8914198c90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 --2- 172.21.15.186:0/2358252153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8914102660 0x7f89141991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 >> 172.21.15.186:0/2358252153 conn(0x7f89140fc710 msgr2=0x7f89140fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:30.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 shutdown_connections 2024-04-25T20:54:30.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:54:30.675+0000 7f8918f6b700 1 -- 172.21.15.186:0/2358252153 wait complete. 2024-04-25T20:54:31.245 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T20:54:31.248 INFO:tasks.cephadm:Deploying osd.9 on smithi047 with /dev/vg_nvme/lv_1... 2024-04-25T20:54:31.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T20:54:31.505 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:31.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:31 smithi047 ceph-mon[29195]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:31.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:31 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:31.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:31 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:31.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:31 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:31.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:31 smithi047 ceph-mon[29195]: from='osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:54:31.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:31 smithi047 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:54:31.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:31 smithi186 ceph-mon[32529]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:31.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:31 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:31.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:31 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:31.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:31 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:31.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:31 smithi186 ceph-mon[32529]: from='osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:54:31.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:31 smithi186 ceph-mon[32529]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:54:32.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:31 smithi092 ceph-mon[32524]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:32.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:31 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:32.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:31 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:32.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:31 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:32.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:31 smithi092 ceph-mon[32524]: from='osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:54:32.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:31 smithi092 ceph-mon[32524]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:54:32.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:32 smithi047 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T20:54:32.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:32 smithi047 ceph-mon[29195]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T20:54:32.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:32.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:32 smithi047 ceph-mon[29195]: from='osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:54:32.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:32 smithi047 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:54:32.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:32 smithi186 ceph-mon[32529]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T20:54:32.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:32 smithi186 ceph-mon[32529]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T20:54:32.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:32.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:32 smithi186 ceph-mon[32529]: from='osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:54:32.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:32 smithi186 ceph-mon[32529]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:54:32.971 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 20:54:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T20:54:32.689+0000 7fc51d526700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T20:54:32.971 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 20:54:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T20:54:32.700+0000 7fc519319700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:54:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:32 smithi092 ceph-mon[32524]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T20:54:33.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:32 smithi092 ceph-mon[32524]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T20:54:33.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:33.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:32 smithi092 ceph-mon[32524]: from='osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:54:33.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:32 smithi092 ceph-mon[32524]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:54:33.306 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T20:54:33.307 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T20:54:33.307 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-04-25T20:54:33.307 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-04-25T20:54:33.307 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0092432 s, 1.1 GB/s 2024-04-25T20:54:33.307 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-04-25T20:54:33.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:33 smithi186 ceph-mon[32529]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:33.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:33 smithi186 ceph-mon[32529]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:54:33.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:33 smithi186 ceph-mon[32529]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T20:54:33.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:33.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:33 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:33.975 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:33 smithi047 ceph-mon[29195]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:33.975 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:33 smithi047 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:54:33.975 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:33 smithi047 ceph-mon[29195]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T20:54:33.975 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:33.975 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:33 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:34.003 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-04-25T20:54:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:33 smithi092 ceph-mon[32524]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:33 smithi092 ceph-mon[32524]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:54:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:33 smithi092 ceph-mon[32524]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T20:54:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:34.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:33 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:34.261 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:54:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:54:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] boot 2024-04-25T20:54:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T20:54:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:34.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:34.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:34.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:34.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:34.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:34.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:34.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:34 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:34.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:54:34.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:54:34.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] boot 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:34 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:35.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: osd.8 [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] boot 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:35.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:35.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:35.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:35.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:34 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3921576287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff68075420 msgr2=0x7eff680758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3921576287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff68075420 0x7eff680758a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009990 tx=0x7eff5c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3921576287 shutdown_connections 2024-04-25T20:54:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3921576287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6810a9a0 0x7eff6810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3921576287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff680769f0 0x7eff68074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3921576287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff68075420 0x7eff680758a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3921576287 >> 172.21.15.47:0/3921576287 conn(0x7eff680fe040 msgr2=0x7eff68100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3921576287 shutdown_connections 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.719+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3921576287 wait complete. 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff6ce9a700 1 Processor -- start 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff6ce9a700 1 -- start start 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff6ce9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68075420 0x7eff6819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff6ce9a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff680769f0 0x7eff6819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff6ce9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6810a9a0 0x7eff681a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff6ce9a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7eff68110750 con 0x7eff680769f0 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff6ce9a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7eff681105d0 con 0x7eff6810a9a0 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff6ce9a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7eff681108d0 con 0x7eff68075420 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.720+0000 7eff65d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff680769f0 0x7eff6819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff65d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff680769f0 0x7eff6819b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50616/0 (socket says 172.21.15.47:50616) 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff65d9b700 1 -- 172.21.15.47:0/3257640913 learned_addr learned my addr 172.21.15.47:0/3257640913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff6659c700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68075420 0x7eff6819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff66d9d700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6810a9a0 0x7eff681a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff65d9b700 1 -- 172.21.15.47:0/3257640913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68075420 msgr2=0x7eff6819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff65d9b700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68075420 0x7eff6819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff65d9b700 1 -- 172.21.15.47:0/3257640913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6810a9a0 msgr2=0x7eff681a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff65d9b700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6810a9a0 0x7eff681a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff65d9b700 1 -- 172.21.15.47:0/3257640913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff681a2970 con 0x7eff680769f0 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff65d9b700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff680769f0 0x7eff6819b260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0036f0 tx=0x7eff5c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff577fe700 1 -- 172.21.15.47:0/3257640913 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c027070 con 0x7eff680769f0 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff577fe700 1 -- 172.21.15.47:0/3257640913 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c004000 con 0x7eff680769f0 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff681a2c00 con 0x7eff680769f0 2024-04-25T20:54:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff577fe700 1 -- 172.21.15.47:0/3257640913 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c022d20 con 0x7eff680769f0 2024-04-25T20:54:35.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.721+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff681a3050 con 0x7eff680769f0 2024-04-25T20:54:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.722+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff68069730 con 0x7eff680769f0 2024-04-25T20:54:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.724+0000 7eff577fe700 1 -- 172.21.15.47:0/3257640913 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff5c02a070 con 0x7eff680769f0 2024-04-25T20:54:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.724+0000 7eff577fe700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7eff500a3c40 0x7eff500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.724+0000 7eff6659c700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7eff500a3c40 0x7eff500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.724+0000 7eff577fe700 1 -- 172.21.15.47:0/3257640913 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7eff5c0d1140 con 0x7eff680769f0 2024-04-25T20:54:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.725+0000 7eff6659c700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7eff500a3c40 0x7eff500a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff6819c200 tx=0x7eff5800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.728+0000 7eff577fe700 1 -- 172.21.15.47:0/3257640913 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff5c031020 con 0x7eff680769f0 2024-04-25T20:54:35.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:35 smithi047 ceph-mon[29195]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:35.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:35 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi186 2024-04-25T20:54:35.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:35 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-25T20:54:35.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:35 smithi047 ceph-mon[29195]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T20:54:35.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:35.948+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7eff68062e80 con 0x7eff500a3c40 2024-04-25T20:54:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:35 smithi186 ceph-mon[32529]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:35 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi186 2024-04-25T20:54:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:35 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-25T20:54:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:35 smithi186 ceph-mon[32529]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T20:54:36.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:35 smithi092 ceph-mon[32524]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:54:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:35 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi186 2024-04-25T20:54:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:35 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-25T20:54:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:35 smithi092 ceph-mon[32524]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T20:54:36.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:36 smithi047 ceph-mon[29195]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T20:54:36.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:36 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:54:36.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:36 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:54:36.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:36 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:36.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:36 smithi047 ceph-mon[29195]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T20:54:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:36 smithi186 ceph-mon[32529]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T20:54:36.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:36 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:54:36.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:36 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:54:36.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:36 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:36.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:36 smithi186 ceph-mon[32529]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T20:54:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:36 smithi092 ceph-mon[32524]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T20:54:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:36 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:54:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:36 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:54:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:36 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:36 smithi092 ceph-mon[32524]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T20:54:37.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:37 smithi047 ceph-mon[29195]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:37.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:37 smithi047 ceph-mon[29195]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:37.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:37 smithi047 ceph-mon[29195]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T20:54:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:37 smithi186 ceph-mon[32529]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:37 smithi186 ceph-mon[32529]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:37 smithi186 ceph-mon[32529]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T20:54:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:37 smithi092 ceph-mon[32524]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:37 smithi092 ceph-mon[32524]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:37 smithi092 ceph-mon[32524]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T20:54:39.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:39 smithi186 ceph-mon[32529]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:39 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3573842054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b313eacf-dc05-4b28-a5d4-4d04e7b1bda9"}]: dispatch 2024-04-25T20:54:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:39 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3573842054' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b313eacf-dc05-4b28-a5d4-4d04e7b1bda9"}]': finished 2024-04-25T20:54:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:39 smithi186 ceph-mon[32529]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T20:54:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:39 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2365002995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:54:40.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:39 smithi092 ceph-mon[32524]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:40.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:39 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3573842054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b313eacf-dc05-4b28-a5d4-4d04e7b1bda9"}]: dispatch 2024-04-25T20:54:40.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:39 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3573842054' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b313eacf-dc05-4b28-a5d4-4d04e7b1bda9"}]': finished 2024-04-25T20:54:40.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:39 smithi092 ceph-mon[32524]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T20:54:40.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:40.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:39 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2365002995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:54:40.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:39 smithi047 ceph-mon[29195]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:40.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:39 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3573842054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b313eacf-dc05-4b28-a5d4-4d04e7b1bda9"}]: dispatch 2024-04-25T20:54:40.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:39 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3573842054' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b313eacf-dc05-4b28-a5d4-4d04e7b1bda9"}]': finished 2024-04-25T20:54:40.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:39 smithi047 ceph-mon[29195]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T20:54:40.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:40.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:39 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2365002995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:54:41.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:41 smithi186 ceph-mon[32529]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-25T20:54:42.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:41 smithi092 ceph-mon[32524]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-25T20:54:42.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:41 smithi047 ceph-mon[29195]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-25T20:54:43.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:43 smithi186 ceph-mon[32529]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:54:43.982 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:43 smithi047 ceph-mon[29195]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:54:44.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:43 smithi092 ceph-mon[32524]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:54:45.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:44 smithi047 ceph-mon[29195]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:54:45.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:44 smithi092 ceph-mon[32524]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:54:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:44 smithi186 ceph-mon[32529]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:54:47.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:46 smithi186 ceph-mon[32529]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:54:47.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T20:54:47.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:47.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:46 smithi186 ceph-mon[32529]: Deploying daemon osd.9 on smithi047 2024-04-25T20:54:47.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:46 smithi092 ceph-mon[32524]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:54:47.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T20:54:47.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:47.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:46 smithi092 ceph-mon[32524]: Deploying daemon osd.9 on smithi047 2024-04-25T20:54:47.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:46 smithi047 ceph-mon[29195]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:54:47.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T20:54:47.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:47.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:46 smithi047 ceph-mon[29195]: Deploying daemon osd.9 on smithi047 2024-04-25T20:54:49.199 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:48 smithi047 ceph-mon[29195]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T20:54:49.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:48 smithi186 ceph-mon[32529]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T20:54:49.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:48 smithi092 ceph-mon[32524]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T20:54:51.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:50 smithi186 ceph-mon[32529]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T20:54:51.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:50 smithi047 ceph-mon[29195]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T20:54:51.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:50 smithi092 ceph-mon[32524]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T20:54:52.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:51 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:52.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:51 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:52.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:51 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:52.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:51 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:52.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:51 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:52.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:51 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:53 smithi047 ceph-mon[29195]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:53.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:53 smithi186 ceph-mon[32529]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:54.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:53 smithi092 ceph-mon[32524]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:55.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.353+0000 7eff577fe700 1 -- 172.21.15.47:0/3257640913 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff68062e80 con 0x7eff500a3c40 2024-04-25T20:54:55.354 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 9 on host 'smithi047' 2024-04-25T20:54:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7eff500a3c40 msgr2=0x7eff500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7eff500a3c40 0x7eff500a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff6819c200 tx=0x7eff5800a380 comp rx=0 tx=0).stop 2024-04-25T20:54:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff680769f0 msgr2=0x7eff6819b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff680769f0 0x7eff6819b260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0036f0 tx=0x7eff5c005f10 comp rx=0 tx=0).stop 2024-04-25T20:54:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 shutdown_connections 2024-04-25T20:54:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7eff500a3c40 0x7eff500a6100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6810a9a0 0x7eff681a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68075420 0x7eff6819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:55.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 --2- 172.21.15.47:0/3257640913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff680769f0 0x7eff6819b260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:55.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 >> 172.21.15.47:0/3257640913 conn(0x7eff680fe040 msgr2=0x7eff681068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:55.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 shutdown_connections 2024-04-25T20:54:55.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:54:55.355+0000 7eff6ce9a700 1 -- 172.21.15.47:0/3257640913 wait complete. 2024-04-25T20:54:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:55.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:55.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:55 smithi047 ceph-mon[29195]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:55.836 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:55 smithi092 ceph-mon[32524]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T20:54:55.882 DEBUG:teuthology.orchestra.run.smithi047:osd.9> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T20:54:55.885 INFO:tasks.cephadm:Deploying osd.10 on smithi092 with /dev/vg_nvme/lv_1... 2024-04-25T20:54:55.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T20:54:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:55.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:55.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:55.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:55.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:55 smithi186 ceph-mon[32529]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T20:54:56.125 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:54:57.513 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:57 smithi092 ceph-mon[32524]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:57.513 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:57 smithi092 ceph-mon[32524]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T20:54:57.513 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:57 smithi092 ceph-mon[32524]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T20:54:57.513 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:57 smithi092 ceph-mon[32524]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:54:57.513 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:57 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:57.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:57 smithi047 ceph-mon[29195]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:57.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:57 smithi047 ceph-mon[29195]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T20:54:57.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:57 smithi047 ceph-mon[29195]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T20:54:57.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:57 smithi047 ceph-mon[29195]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:54:57.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:57 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:57.612 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 20:54:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T20:54:57.353+0000 7fdf53c3c700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T20:54:57.612 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 20:54:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T20:54:57.368+0000 7fdf4fa2f700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:54:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:57 smithi186 ceph-mon[32529]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:57 smithi186 ceph-mon[32529]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T20:54:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:57 smithi186 ceph-mon[32529]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T20:54:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:57 smithi186 ceph-mon[32529]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-25T20:54:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:57 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:58.182 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T20:54:58.182 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T20:54:58.182 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-04-25T20:54:58.182 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-04-25T20:54:58.183 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116897 s, 897 MB/s 2024-04-25T20:54:58.183 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-04-25T20:54:58.433 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:58 smithi047 ceph-mon[29195]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:54:58.433 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:58 smithi047 ceph-mon[29195]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T20:54:58.433 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:58.433 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:58 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:58.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:58 smithi092 ceph-mon[32524]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:54:58.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:58 smithi092 ceph-mon[32524]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T20:54:58.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:58.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:58 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:58 smithi186 ceph-mon[32529]: from='osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-25T20:54:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:58 smithi186 ceph-mon[32529]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T20:54:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:58 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:58.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-04-25T20:54:59.058 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] boot 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.454 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.455 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.455 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.455 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.455 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.455 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:59.455 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:59.455 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:54:59 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:54:59.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:54:59.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:59.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] boot 2024-04-25T20:54:59.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:54:59 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: osd.9 [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] boot 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:59.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:59.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:54:59 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 -- 172.21.15.92:0/3271853957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0481046f0 msgr2=0x7fe04810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/3271853957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0481046f0 0x7fe04810afa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe040009990 tx=0x7fe040009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 -- 172.21.15.92:0/3271853957 shutdown_connections 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/3271853957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0481046f0 0x7fe04810afa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/3271853957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe048100d70 0x7fe048101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/3271853957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe048103d50 0x7fe0481041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 -- 172.21.15.92:0/3271853957 >> 172.21.15.92:0/3271853957 conn(0x7fe0480fc6b0 msgr2=0x7fe0480feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 -- 172.21.15.92:0/3271853957 shutdown_connections 2024-04-25T20:55:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.517+0000 7fe04e0c0700 1 -- 172.21.15.92:0/3271853957 wait complete. 2024-04-25T20:55:00.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:00 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi047 2024-04-25T20:55:00.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:00 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi047 to 4273M 2024-04-25T20:55:00.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:00 smithi092 ceph-mon[32524]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe04e0c0700 1 Processor -- start 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe04e0c0700 1 -- start start 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe04e0c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe048100d70 0x7fe048198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe04e0c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe048103d50 0x7fe048198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe04e0c0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0481046f0 0x7fe0481a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe04e0c0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe04810e520 con 0x7fe048103d50 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe04e0c0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe04810e3a0 con 0x7fe048100d70 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe04e0c0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe04810e6a0 con 0x7fe0481046f0 2024-04-25T20:55:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe0477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe048100d70 0x7fe048198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe0477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe048100d70 0x7fe048198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38010/0 (socket says 172.21.15.92:38010) 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe0477fe700 1 -- 172.21.15.92:0/712230363 learned_addr learned my addr 172.21.15.92:0/712230363 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.518+0000 7fe047fff700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0481046f0 0x7fe0481a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe046ffd700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe048103d50 0x7fe048198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe0477fe700 1 -- 172.21.15.92:0/712230363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0481046f0 msgr2=0x7fe0481a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe0477fe700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0481046f0 0x7fe0481a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe0477fe700 1 -- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe048103d50 msgr2=0x7fe048198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe0477fe700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe048103d50 0x7fe048198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe0477fe700 1 -- 172.21.15.92:0/712230363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0481a0710 con 0x7fe048100d70 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe047fff700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0481046f0 0x7fe0481a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe046ffd700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe048103d50 0x7fe048198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:55:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe0477fe700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe048100d70 0x7fe048198ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe038007e00 tx=0x7fe03800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe044ff9700 1 -- 172.21.15.92:0/712230363 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038016020 con 0x7fe048100d70 2024-04-25T20:55:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe044ff9700 1 -- 172.21.15.92:0/712230363 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe03800ad40 con 0x7fe048100d70 2024-04-25T20:55:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0481a0a00 con 0x7fe048100d70 2024-04-25T20:55:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe044ff9700 1 -- 172.21.15.92:0/712230363 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038011700 con 0x7fe048100d70 2024-04-25T20:55:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.519+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0481a0f40 con 0x7fe048100d70 2024-04-25T20:55:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.520+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe048069730 con 0x7fe048100d70 2024-04-25T20:55:00.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.521+0000 7fe044ff9700 1 -- 172.21.15.92:0/712230363 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe038019020 con 0x7fe048100d70 2024-04-25T20:55:00.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.522+0000 7fe044ff9700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe0300a3c40 0x7fe0300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:00.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.522+0000 7fe044ff9700 1 -- 172.21.15.92:0/712230363 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fe0380cfc80 con 0x7fe048100d70 2024-04-25T20:55:00.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.522+0000 7fe046ffd700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe0300a3c40 0x7fe0300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:00.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.523+0000 7fe046ffd700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe0300a3c40 0x7fe0300a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe03c005fd0 tx=0x7fe03c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:00.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.526+0000 7fe044ff9700 1 -- 172.21.15.92:0/712230363 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe038020030 con 0x7fe048100d70 2024-04-25T20:55:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:00 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi047 2024-04-25T20:55:00.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:00 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi047 to 4273M 2024-04-25T20:55:00.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:00 smithi186 ceph-mon[32529]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T20:55:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:00.730+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe048062e80 con 0x7fe0300a3c40 2024-04-25T20:55:00.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:00 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi047 2024-04-25T20:55:00.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:00 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi047 to 4273M 2024-04-25T20:55:00.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:00 smithi047 ceph-mon[29195]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T20:55:01.601 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:01 smithi092 ceph-mon[32524]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:01.601 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:55:01.601 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:55:01.601 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:01 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:01.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:01 smithi186 ceph-mon[32529]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:01.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:55:01.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:55:01.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:01 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:01 smithi047 ceph-mon[29195]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:55:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:55:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:01 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:02 smithi092 ceph-mon[32524]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:02.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:02 smithi186 ceph-mon[32529]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:02.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:02 smithi047 ceph-mon[29195]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:03.718 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:03 smithi092 ceph-mon[32524]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:03 smithi186 ceph-mon[32529]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:03.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:03 smithi047 ceph-mon[29195]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:04 smithi186 ceph-mon[32529]: from='client.? 172.21.15.92:0/2304955488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]: dispatch 2024-04-25T20:55:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:04 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]: dispatch 2024-04-25T20:55:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:04 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]': finished 2024-04-25T20:55:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:04 smithi186 ceph-mon[32529]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T20:55:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:04 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:04 smithi186 ceph-mon[32529]: from='client.? 172.21.15.92:0/388540445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:55:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:04 smithi092 ceph-mon[32524]: from='client.? 172.21.15.92:0/2304955488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]: dispatch 2024-04-25T20:55:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:04 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]: dispatch 2024-04-25T20:55:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:04 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]': finished 2024-04-25T20:55:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:04 smithi092 ceph-mon[32524]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T20:55:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:04 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:04 smithi092 ceph-mon[32524]: from='client.? 172.21.15.92:0/388540445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:55:04.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:04 smithi047 ceph-mon[29195]: from='client.? 172.21.15.92:0/2304955488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]: dispatch 2024-04-25T20:55:04.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:04 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]: dispatch 2024-04-25T20:55:04.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:04 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a31b12e-d889-4a51-9060-b79255c81626"}]': finished 2024-04-25T20:55:04.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:04 smithi047 ceph-mon[29195]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T20:55:04.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:04 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:04.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:04 smithi047 ceph-mon[29195]: from='client.? 172.21.15.92:0/388540445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:55:05.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:05 smithi092 ceph-mon[32524]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:05 smithi047 ceph-mon[29195]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:05 smithi186 ceph-mon[32529]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:07.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:07 smithi092 ceph-mon[32524]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:07.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:07 smithi047 ceph-mon[29195]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:07.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:07 smithi186 ceph-mon[32529]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:09.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:09 smithi092 ceph-mon[32524]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:09 smithi047 ceph-mon[29195]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:09.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:09 smithi186 ceph-mon[32529]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:11.763 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:11 smithi092 ceph-mon[32524]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:11.763 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T20:55:11.763 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:11.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:11 smithi047 ceph-mon[29195]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:11.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T20:55:11.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:11.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:11 smithi186 ceph-mon[32529]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T20:55:11.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:12.694 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:12 smithi092 ceph-mon[32524]: Deploying daemon osd.10 on smithi092 2024-04-25T20:55:12.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:12 smithi047 ceph-mon[29195]: Deploying daemon osd.10 on smithi092 2024-04-25T20:55:12.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:12 smithi186 ceph-mon[32529]: Deploying daemon osd.10 on smithi092 2024-04-25T20:55:13.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:13 smithi092 ceph-mon[32524]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:13.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:13 smithi047 ceph-mon[29195]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:13.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:13 smithi186 ceph-mon[32529]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:15.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:15 smithi092 ceph-mon[32524]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:15 smithi047 ceph-mon[29195]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:15 smithi186 ceph-mon[32529]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:17 smithi092 ceph-mon[32524]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:18.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:17 smithi047 ceph-mon[29195]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:18.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:18.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:18.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:18.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:17 smithi186 ceph-mon[32529]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:18.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:18.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:18.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:18.915 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:18 smithi092 ceph-mon[32524]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:19.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:18 smithi047 ceph-mon[29195]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:18 smithi186 ceph-mon[32529]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:20.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.400+0000 7fe044ff9700 1 -- 172.21.15.92:0/712230363 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe048062e80 con 0x7fe0300a3c40 2024-04-25T20:55:20.402 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 10 on host 'smithi092' 2024-04-25T20:55:20.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe0300a3c40 msgr2=0x7fe0300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:20.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe0300a3c40 0x7fe0300a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe03c005fd0 tx=0x7fe03c005ee0 comp rx=0 tx=0).stop 2024-04-25T20:55:20.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe048100d70 msgr2=0x7fe048198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe048100d70 0x7fe048198ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe038007e00 tx=0x7fe03800a340 comp rx=0 tx=0).stop 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 shutdown_connections 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe0300a3c40 0x7fe0300a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0481046f0 0x7fe0481a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe048100d70 0x7fe048198ab0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.402+0000 7fe04e0c0700 1 --2- 172.21.15.92:0/712230363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe048103d50 0x7fe048198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.403+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 >> 172.21.15.92:0/712230363 conn(0x7fe0480fc6b0 msgr2=0x7fe048107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.403+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 shutdown_connections 2024-04-25T20:55:20.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-25T20:55:20.403+0000 7fe04e0c0700 1 -- 172.21.15.92:0/712230363 wait complete. 2024-04-25T20:55:20.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:20.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:20.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:20.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:21.038 DEBUG:teuthology.orchestra.run.smithi092:osd.10> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T20:55:21.041 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2024-04-25T20:55:21.041 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T20:55:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:21.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:21.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:21.290 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:55:21.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:21 smithi186 ceph-mon[32529]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:21.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:21 smithi186 ceph-mon[32529]: from='osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:55:21.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:21 smithi186 ceph-mon[32529]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:55:22.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:21 smithi092 ceph-mon[32524]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:22.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:21 smithi092 ceph-mon[32524]: from='osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:55:22.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:21 smithi092 ceph-mon[32524]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:55:22.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:21 smithi047 ceph-mon[29195]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:22.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:21 smithi047 ceph-mon[29195]: from='osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:55:22.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:21 smithi047 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:55:22.682 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:22 smithi186 ceph-mon[32529]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T20:55:22.682 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:22 smithi186 ceph-mon[32529]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T20:55:22.682 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:22 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:22.682 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:22 smithi186 ceph-mon[32529]: from='osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:55:22.682 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:22 smithi186 ceph-mon[32529]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:55:22.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:22 smithi092 ceph-mon[32524]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T20:55:22.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:22 smithi092 ceph-mon[32524]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T20:55:22.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:22 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:22.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:22 smithi092 ceph-mon[32524]: from='osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:55:22.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:22 smithi092 ceph-mon[32524]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:55:23.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:22 smithi047 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T20:55:23.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:22 smithi047 ceph-mon[29195]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T20:55:23.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:22 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:23.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:22 smithi047 ceph-mon[29195]: from='osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:55:23.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:22 smithi047 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-04-25T20:55:23.250 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T20:55:23.250 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T20:55:23.250 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-25T20:55:23.250 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-25T20:55:23.250 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115975 s, 904 MB/s 2024-04-25T20:55:23.250 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-25T20:55:23.684 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:23 smithi092 ceph-mon[32524]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:23.685 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:23 smithi092 ceph-mon[32524]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:55:23.685 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:23 smithi092 ceph-mon[32524]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T20:55:23.685 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:23.685 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:23.685 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:23 smithi092 ceph-mon[32524]: from='osd.10 ' entity='osd.10' 2024-04-25T20:55:23.685 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 20:55:23 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T20:55:23.582+0000 7f6b4ae55700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T20:55:23.685 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 20:55:23 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T20:55:23.683+0000 7f6b42c40700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:55:23.919 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:23 smithi186 ceph-mon[32529]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:23.919 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:23 smithi186 ceph-mon[32529]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:55:23.919 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:23 smithi186 ceph-mon[32529]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T20:55:23.919 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:23.919 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:23.919 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:23 smithi186 ceph-mon[32529]: from='osd.10 ' entity='osd.10' 2024-04-25T20:55:23.950 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2024-04-25T20:55:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:23 smithi047 ceph-mon[29195]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:23 smithi047 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-04-25T20:55:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:23 smithi047 ceph-mon[29195]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T20:55:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:24.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:23 smithi047 ceph-mon[29195]: from='osd.10 ' entity='osd.10' 2024-04-25T20:55:24.202 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:55:24.958 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:24.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:24.960 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:24.960 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:24.960 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:24 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:25.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:55:25.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:55:25.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:25.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T20:55:25.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:25.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:25.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:24 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:25.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:55:25.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:25.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:25.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:25.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:24 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:25.855 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 -- 172.21.15.186:0/2315617677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 msgr2=0x7f4d38109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:25.855 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/2315617677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 0x7f4d38109de0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000b0a0 tx=0x7f4d3000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.855 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 -- 172.21.15.186:0/2315617677 shutdown_connections 2024-04-25T20:55:25.855 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/2315617677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 0x7f4d38109de0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.855 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/2315617677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d38101990 0x7f4d38101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.855 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/2315617677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38102b90 0x7f4d38102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 -- 172.21.15.186:0/2315617677 >> 172.21.15.186:0/2315617677 conn(0x7f4d380fcf30 msgr2=0x7f4d380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 -- 172.21.15.186:0/2315617677 shutdown_connections 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.849+0000 7f4d3fb01700 1 -- 172.21.15.186:0/2315617677 wait complete. 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3fb01700 1 Processor -- start 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3fb01700 1 -- start start 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3fb01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d38101990 0x7f4d3806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3fb01700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38102b90 0x7f4d3819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3fb01700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 0x7f4d3819ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3fb01700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810d480 con 0x7f4d38102b90 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3fb01700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810d300 con 0x7f4d38101990 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3fb01700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810d600 con 0x7f4d38103530 2024-04-25T20:55:25.856 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 0x7f4d3819ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 0x7f4d3819ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:49002/0 (socket says 172.21.15.186:49002) 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 -- 172.21.15.186:0/3480774923 learned_addr learned my addr 172.21.15.186:0/3480774923 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3d89d700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d38101990 0x7f4d3806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3d09c700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38102b90 0x7f4d3819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 -- 172.21.15.186:0/3480774923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d38101990 msgr2=0x7f4d3806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d38101990 0x7f4d3806bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 -- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38102b90 msgr2=0x7f4d3819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38102b90 0x7f4d3819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 -- 172.21.15.186:0/3480774923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d381a0800 con 0x7f4d38103530 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3d09c700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38102b90 0x7f4d3819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.850+0000 7f4d3e09e700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 0x7f4d3819ff30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000b070 tx=0x7f4d30004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.851+0000 7f4d2effd700 1 -- 172.21.15.186:0/3480774923 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30021030 con 0x7f4d38103530 2024-04-25T20:55:25.857 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.851+0000 7f4d2effd700 1 -- 172.21.15.186:0/3480774923 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d30003a00 con 0x7f4d38103530 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.851+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d381a0a90 con 0x7f4d38103530 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.851+0000 7f4d2effd700 1 -- 172.21.15.186:0/3480774923 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30004600 con 0x7f4d38103530 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.851+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d381a0f40 con 0x7f4d38103530 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.852+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d38069730 con 0x7f4d38103530 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.853+0000 7f4d2effd700 1 -- 172.21.15.186:0/3480774923 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4d30024070 con 0x7f4d38103530 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.853+0000 7f4d2effd700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d240a3c40 0x7f4d240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.854+0000 7f4d2effd700 1 -- 172.21.15.186:0/3480774923 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f4d3000f020 con 0x7f4d38103530 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.854+0000 7f4d3d89d700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d240a3c40 0x7f4d240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.854+0000 7f4d3d89d700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d240a3c40 0x7f4d240a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d381029f0 tx=0x7f4d34007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:25.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:25.857+0000 7f4d2effd700 1 -- 172.21.15.186:0/3480774923 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d3009e0e0 con 0x7f4d38103530 2024-04-25T20:55:25.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:25 smithi186 ceph-mon[32529]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:25.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:25 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi092 2024-04-25T20:55:25.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:25 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi092 to 4272M 2024-04-25T20:55:25.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:25 smithi186 ceph-mon[32529]: osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] boot 2024-04-25T20:55:25.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:25 smithi186 ceph-mon[32529]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T20:55:25.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:25 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:25 smithi092 ceph-mon[32524]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:25 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi092 2024-04-25T20:55:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:25 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi092 to 4272M 2024-04-25T20:55:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:25 smithi092 ceph-mon[32524]: osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] boot 2024-04-25T20:55:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:25 smithi092 ceph-mon[32524]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T20:55:26.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:25 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:26.067 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:26.067+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4d38062e80 con 0x7f4d240a3c40 2024-04-25T20:55:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:25 smithi047 ceph-mon[29195]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:55:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:25 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi092 2024-04-25T20:55:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:25 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi092 to 4272M 2024-04-25T20:55:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:25 smithi047 ceph-mon[29195]: osd.10 [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] boot 2024-04-25T20:55:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:25 smithi047 ceph-mon[29195]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T20:55:26.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:55:26.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:26 smithi186 ceph-mon[32529]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T20:55:26.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:26 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:55:26.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:26 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:55:26.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:26 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:27.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:26 smithi092 ceph-mon[32524]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T20:55:27.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:26 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:55:27.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:26 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:55:27.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:26 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:26 smithi047 ceph-mon[29195]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T20:55:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:26 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:55:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:26 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:55:27.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:26 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:27.846 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:27 smithi186 ceph-mon[32529]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:27.846 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:27 smithi186 ceph-mon[32529]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:27.846 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:27 smithi186 ceph-mon[32529]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T20:55:28.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:27 smithi092 ceph-mon[32524]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:28.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:27 smithi092 ceph-mon[32524]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:28.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:27 smithi092 ceph-mon[32524]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T20:55:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:27 smithi047 ceph-mon[29195]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:27 smithi047 ceph-mon[29195]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:27 smithi047 ceph-mon[29195]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T20:55:28.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:28 smithi186 ceph-mon[32529]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T20:55:28.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:28 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/3670647967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]: dispatch 2024-04-25T20:55:28.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:28 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]: dispatch 2024-04-25T20:55:28.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:28 smithi186 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]': finished 2024-04-25T20:55:29.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:28 smithi092 ceph-mon[32524]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T20:55:29.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:28 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/3670647967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]: dispatch 2024-04-25T20:55:29.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:28 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]: dispatch 2024-04-25T20:55:29.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:28 smithi092 ceph-mon[32524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]': finished 2024-04-25T20:55:29.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:28 smithi047 ceph-mon[29195]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T20:55:29.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:28 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/3670647967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]: dispatch 2024-04-25T20:55:29.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:28 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]: dispatch 2024-04-25T20:55:29.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:28 smithi047 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92dc80e5-665e-412c-9596-b4ea0f65e561"}]': finished 2024-04-25T20:55:29.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:29 smithi186 ceph-mon[32529]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:29.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:29 smithi186 ceph-mon[32529]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T20:55:29.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:29 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:29.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:29 smithi186 ceph-mon[32529]: from='client.? 172.21.15.186:0/1556795092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:55:30.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:29 smithi092 ceph-mon[32524]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:30.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:29 smithi092 ceph-mon[32524]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T20:55:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:29 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:30.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:29 smithi092 ceph-mon[32524]: from='client.? 172.21.15.186:0/1556795092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:55:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:29 smithi047 ceph-mon[29195]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:29 smithi047 ceph-mon[29195]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T20:55:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:29 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:30.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:29 smithi047 ceph-mon[29195]: from='client.? 172.21.15.186:0/1556795092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:55:31.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:31 smithi186 ceph-mon[32529]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:55:32.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:31 smithi092 ceph-mon[32524]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:55:32.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:31 smithi047 ceph-mon[29195]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:55:33.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:33 smithi186 ceph-mon[32529]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:55:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:33 smithi092 ceph-mon[32524]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:55:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:33 smithi047 ceph-mon[29195]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:55:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:35 smithi186 ceph-mon[32529]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:55:36.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:35 smithi092 ceph-mon[32524]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:55:36.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:35 smithi047 ceph-mon[29195]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:55:37.007 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:36 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T20:55:37.008 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:36 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:37.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:36 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T20:55:37.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:36 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:36 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T20:55:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:36 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:37 smithi186 ceph-mon[32529]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:55:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:37 smithi186 ceph-mon[32529]: Deploying daemon osd.11 on smithi186 2024-04-25T20:55:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:37 smithi092 ceph-mon[32524]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:55:38.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:37 smithi092 ceph-mon[32524]: Deploying daemon osd.11 on smithi186 2024-04-25T20:55:38.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:37 smithi047 ceph-mon[29195]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:55:38.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:37 smithi047 ceph-mon[29195]: Deploying daemon osd.11 on smithi186 2024-04-25T20:55:38.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:38 smithi186 ceph-mon[32529]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T20:55:39.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:38 smithi092 ceph-mon[32524]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T20:55:39.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:38 smithi047 ceph-mon[29195]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T20:55:41.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:40 smithi186 ceph-mon[32529]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T20:55:41.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:40 smithi092 ceph-mon[32524]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T20:55:41.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:40 smithi047 ceph-mon[29195]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T20:55:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:42 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:42 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:42 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:43.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:42 smithi047 ceph-mon[29195]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:42 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:42 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:42 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:42 smithi186 ceph-mon[32529]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:43.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:42 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:43.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:42 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:43.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:42 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:43.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:42 smithi092 ceph-mon[32524]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:45.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:44 smithi186 ceph-mon[32529]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:45.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:44 smithi092 ceph-mon[32524]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:45.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.287+0000 7f4d2effd700 1 -- 172.21.15.186:0/3480774923 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4d38062e80 con 0x7f4d240a3c40 2024-04-25T20:55:45.289 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d240a3c40 msgr2=0x7f4d240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d240a3c40 0x7f4d240a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d381029f0 tx=0x7f4d34007400 comp rx=0 tx=0).stop 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 msgr2=0x7f4d3819ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 0x7f4d3819ff30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000b070 tx=0x7f4d30004e40 comp rx=0 tx=0).stop 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 shutdown_connections 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d240a3c40 0x7f4d240a6100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d38103530 0x7f4d3819ff30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d38101990 0x7f4d3806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:45.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 --2- 172.21.15.186:0/3480774923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38102b90 0x7f4d3819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:45.292 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 >> 172.21.15.186:0/3480774923 conn(0x7f4d380fcf30 msgr2=0x7f4d38105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:45.292 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 shutdown_connections 2024-04-25T20:55:45.292 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:45.290+0000 7f4d3fb01700 1 -- 172.21.15.186:0/3480774923 wait complete. 2024-04-25T20:55:45.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:44 smithi047 ceph-mon[29195]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:45.906 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T20:55:45.909 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T20:55:45.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T20:55:46.177 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:46.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:46 smithi092 ceph-mon[32524]: from='osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:55:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:46 smithi092 ceph-mon[32524]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:55:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:46 smithi047 ceph-mon[29195]: from='osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:55:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:46 smithi047 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:55:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.662 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.662 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.662 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:46 smithi186 ceph-mon[32529]: from='osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:55:46.662 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:46 smithi186 ceph-mon[32529]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:55:46.662 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:46.662 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:47.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:47 smithi092 ceph-mon[32524]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:47 smithi092 ceph-mon[32524]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T20:55:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:47 smithi092 ceph-mon[32524]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T20:55:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:47 smithi092 ceph-mon[32524]: from='osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:55:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:47 smithi092 ceph-mon[32524]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:55:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:47 smithi047 ceph-mon[29195]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:47 smithi047 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T20:55:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:47 smithi047 ceph-mon[29195]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T20:55:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:47 smithi047 ceph-mon[29195]: from='osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:55:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:47 smithi047 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:55:47.634 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:47 smithi186 ceph-mon[32529]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:47.634 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:47 smithi186 ceph-mon[32529]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T20:55:47.634 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:47 smithi186 ceph-mon[32529]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T20:55:47.634 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:47.634 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:47 smithi186 ceph-mon[32529]: from='osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:55:47.634 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:47 smithi186 ceph-mon[32529]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-25T20:55:47.634 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 20:55:47 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T20:55:47.277+0000 7f5f93e34700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T20:55:47.634 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 20:55:47 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T20:55:47.298+0000 7f5f8fc27700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:55:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 -- 172.21.15.47:0/4104299685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 msgr2=0x7fdd9c105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 --2- 172.21.15.47:0/4104299685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 0x7fdd9c105940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009990 tx=0x7fdd98009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 -- 172.21.15.47:0/4104299685 shutdown_connections 2024-04-25T20:55:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 --2- 172.21.15.47:0/4104299685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 0x7fdd9c105940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 --2- 172.21.15.47:0/4104299685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd9c1087b0 0x7fdd9c10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 --2- 172.21.15.47:0/4104299685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd9c105e80 0x7fdd9c108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 -- 172.21.15.47:0/4104299685 >> 172.21.15.47:0/4104299685 conn(0x7fdd9c0fcf60 msgr2=0x7fdd9c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 -- 172.21.15.47:0/4104299685 shutdown_connections 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 -- 172.21.15.47:0/4104299685 wait complete. 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 Processor -- start 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.831+0000 7fdda3f56700 1 -- start start 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda3f56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 0x7fdd9c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda3f56700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd9c105e80 0x7fdd9c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda3f56700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd9c1087b0 0x7fdd9c1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda3f56700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c10e560 con 0x7fdd9c103550 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda3f56700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c10e3e0 con 0x7fdd9c105e80 2024-04-25T20:55:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda3f56700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c10e6e0 con 0x7fdd9c1087b0 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 0x7fdd9c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 0x7fdd9c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47006/0 (socket says 172.21.15.47:47006) 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 -- 172.21.15.47:0/3345220837 learned_addr learned my addr 172.21.15.47:0/3345220837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda14f1700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd9c105e80 0x7fdd9c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda24f3700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd9c1087b0 0x7fdd9c1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 -- 172.21.15.47:0/3345220837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd9c1087b0 msgr2=0x7fdd9c1a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd9c1087b0 0x7fdd9c1a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 -- 172.21.15.47:0/3345220837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd9c105e80 msgr2=0x7fdd9c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd9c105e80 0x7fdd9c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 -- 172.21.15.47:0/3345220837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c1a29f0 con 0x7fdd9c103550 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda1cf2700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 0x7fdd9c19ada0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006010 tx=0x7fdd9800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdd92ffd700 1 -- 172.21.15.47:0/3345220837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98027070 con 0x7fdd9c103550 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdd92ffd700 1 -- 172.21.15.47:0/3345220837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd9800fc60 con 0x7fdd9c103550 2024-04-25T20:55:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdd92ffd700 1 -- 172.21.15.47:0/3345220837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98017650 con 0x7fdd9c103550 2024-04-25T20:55:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c1a2c80 con 0x7fdd9c103550 2024-04-25T20:55:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.832+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c1a30d0 con 0x7fdd9c103550 2024-04-25T20:55:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.833+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9c069730 con 0x7fdd9c103550 2024-04-25T20:55:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.835+0000 7fdd92ffd700 1 -- 172.21.15.47:0/3345220837 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd98009d10 con 0x7fdd9c103550 2024-04-25T20:55:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.835+0000 7fdd92ffd700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd880a3c40 0x7fdd880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.835+0000 7fdda14f1700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd880a3c40 0x7fdd880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.835+0000 7fdd92ffd700 1 -- 172.21.15.47:0/3345220837 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7fdd980d0c00 con 0x7fdd9c103550 2024-04-25T20:55:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.836+0000 7fdda14f1700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd880a3c40 0x7fdd880a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009bf0 tx=0x7fdd8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:47.840+0000 7fdd92ffd700 1 -- 172.21.15.47:0/3345220837 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd9805dff0 con 0x7fdd9c103550 2024-04-25T20:55:48.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.048+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdd9c063650 con 0x7fdd9c103550 2024-04-25T20:55:48.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.049+0000 7fdd92ffd700 1 -- 172.21.15.47:0/3345220837 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7fdd980d7080 con 0x7fdd9c103550 2024-04-25T20:55:48.050 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd880a3c40 msgr2=0x7fdd880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd880a3c40 0x7fdd880a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009bf0 tx=0x7fdd8c009260 comp rx=0 tx=0).stop 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 msgr2=0x7fdd9c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 0x7fdd9c19ada0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006010 tx=0x7fdd9800f510 comp rx=0 tx=0).stop 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 shutdown_connections 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd880a3c40 0x7fdd880a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd9c103550 0x7fdd9c19ada0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd9c1087b0 0x7fdd9c1a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 --2- 172.21.15.47:0/3345220837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd9c105e80 0x7fdd9c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 >> 172.21.15.47:0/3345220837 conn(0x7fdd9c0fcf60 msgr2=0x7fdd9c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 shutdown_connections 2024-04-25T20:55:48.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:48.051+0000 7fdda3f56700 1 -- 172.21.15.47:0/3345220837 wait complete. 2024-04-25T20:55:48.350 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:48 smithi047 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:55:48.350 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:48 smithi047 ceph-mon[29195]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T20:55:48.350 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:48 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:48.350 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:48 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:48.350 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:48 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3345220837' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:55:48.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:48 smithi092 ceph-mon[32524]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:55:48.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:48 smithi092 ceph-mon[32524]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T20:55:48.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:48 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:48.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:48 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:48.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:48 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3345220837' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:55:48.568 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:48 smithi186 ceph-mon[32529]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-25T20:55:48.568 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:48 smithi186 ceph-mon[32529]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T20:55:48.568 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:48 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:48.568 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:48 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:48.568 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:48 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3345220837' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:55:48.825 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1714078524,"num_in_osds":12,"osd_in_since":1714078528,"num_remapped_pgs":0} 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: purged_snaps scrub starts 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: purged_snaps scrub ok 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] boot 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:49.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:49.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: purged_snaps scrub starts 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: purged_snaps scrub ok 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] boot 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:49.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:49.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:55:49.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:55:49.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:55:49.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:49.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: osd.11 [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] boot 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:49.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:49.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T20:55:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:50.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:50 smithi047 ceph-mon[29195]: Detected new or changed devices on smithi186 2024-04-25T20:55:50.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:50 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-25T20:55:50.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:50 smithi047 ceph-mon[29195]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T20:55:50.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:50.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:50.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:50.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:50 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:50 smithi186 ceph-mon[32529]: Detected new or changed devices on smithi186 2024-04-25T20:55:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:50 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-25T20:55:50.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:50 smithi186 ceph-mon[32529]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T20:55:50.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:50.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:50.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:50.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:50 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:50 smithi092 ceph-mon[32524]: Detected new or changed devices on smithi186 2024-04-25T20:55:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:50 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-25T20:55:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:50 smithi092 ceph-mon[32524]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T20:55:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:50.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:50 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:55:51.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:51 smithi047 ceph-mon[29195]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 -- 172.21.15.47:0/299126714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 msgr2=0x7fa608109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/299126714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 0x7fa608109df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa60000b0d0 tx=0x7fa60000b3e0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 -- 172.21.15.47:0/299126714 shutdown_connections 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/299126714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 0x7fa608109df0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/299126714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081019a0 0x7fa608101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/299126714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa608102ba0 0x7fa608103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 -- 172.21.15.47:0/299126714 >> 172.21.15.47:0/299126714 conn(0x7fa6080fcf40 msgr2=0x7fa6080ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 -- 172.21.15.47:0/299126714 shutdown_connections 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 -- 172.21.15.47:0/299126714 wait complete. 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 Processor -- start 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.672+0000 7fa60fb3c700 1 -- start start 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60fb3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081019a0 0x7fa608198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60fb3c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa608102ba0 0x7fa6081990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60fb3c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 0x7fa6081a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60fb3c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810d500 con 0x7fa608103540 2024-04-25T20:55:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60fb3c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810d380 con 0x7fa6081019a0 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60fb3c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810d680 con 0x7fa608102ba0 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 0x7fa6081a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 0x7fa6081a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47024/0 (socket says 172.21.15.47:47024) 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 -- 172.21.15.47:0/3431466090 learned_addr learned my addr 172.21.15.47:0/3431466090 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60d0d7700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa608102ba0 0x7fa6081990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60d8d8700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081019a0 0x7fa608198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 -- 172.21.15.47:0/3431466090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa608102ba0 msgr2=0x7fa6081990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa608102ba0 0x7fa6081990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 -- 172.21.15.47:0/3431466090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081019a0 msgr2=0x7fa608198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081019a0 0x7fa608198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 -- 172.21.15.47:0/3431466090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6081a07e0 con 0x7fa608103540 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.673+0000 7fa60e0d9700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 0x7fa6081a0140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa600004d80 tx=0x7fa600004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.674+0000 7fa5feffd700 1 -- 172.21.15.47:0/3431466090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60001f030 con 0x7fa608103540 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.674+0000 7fa5feffd700 1 -- 172.21.15.47:0/3431466090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa600005040 con 0x7fa608103540 2024-04-25T20:55:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.674+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6081a0a70 con 0x7fa608103540 2024-04-25T20:55:51.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.674+0000 7fa5feffd700 1 -- 172.21.15.47:0/3431466090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60001b690 con 0x7fa608103540 2024-04-25T20:55:51.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.674+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6081a0ef0 con 0x7fa608103540 2024-04-25T20:55:51.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.675+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa608069730 con 0x7fa608103540 2024-04-25T20:55:51.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.676+0000 7fa5feffd700 1 -- 172.21.15.47:0/3431466090 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa600022030 con 0x7fa608103540 2024-04-25T20:55:51.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.677+0000 7fa5feffd700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa5f40a3c40 0x7fa5f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.677+0000 7fa60d8d8700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa5f40a3c40 0x7fa5f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.677+0000 7fa5feffd700 1 -- 172.21.15.47:0/3431466090 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa6000d0e70 con 0x7fa608103540 2024-04-25T20:55:51.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.678+0000 7fa60d8d8700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa5f40a3c40 0x7fa5f40a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa608102a00 tx=0x7fa604009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.681+0000 7fa5feffd700 1 -- 172.21.15.47:0/3431466090 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa60009dd80 con 0x7fa608103540 2024-04-25T20:55:51.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:51 smithi186 ceph-mon[32529]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:51.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:51 smithi092 ceph-mon[32524]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.893+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa6081a13e0 con 0x7fa608103540 2024-04-25T20:55:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.893+0000 7fa5feffd700 1 -- 172.21.15.47:0/3431466090 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7fa60009d680 con 0x7fa608103540 2024-04-25T20:55:51.894 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:55:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa5f40a3c40 msgr2=0x7fa5f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa5f40a3c40 0x7fa5f40a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa608102a00 tx=0x7fa604009190 comp rx=0 tx=0).stop 2024-04-25T20:55:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 msgr2=0x7fa6081a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 0x7fa6081a0140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa600004d80 tx=0x7fa600004db0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 shutdown_connections 2024-04-25T20:55:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa5f40a3c40 0x7fa5f40a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa608103540 0x7fa6081a0140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081019a0 0x7fa608198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 --2- 172.21.15.47:0/3431466090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa608102ba0 0x7fa6081990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 >> 172.21.15.47:0/3431466090 conn(0x7fa6080fcf40 msgr2=0x7fa60806b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 shutdown_connections 2024-04-25T20:55:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:51.895+0000 7fa60fb3c700 1 -- 172.21.15.47:0/3431466090 wait complete. 2024-04-25T20:55:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1714078548,"num_in_osds":12,"osd_in_since":1714078528,"num_remapped_pgs":0} 2024-04-25T20:55:52.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:55:52.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:52 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3431466090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:55:52.687 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:52 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3431466090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:55:52.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:52 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3431466090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:55:53.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:53 smithi186 ceph-mon[32529]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:53.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:53 smithi092 ceph-mon[32524]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:53.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:53 smithi047 ceph-mon[29195]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 -- 172.21.15.47:0/611636652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604102300 msgr2=0x7f660410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 --2- 172.21.15.47:0/611636652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604102300 0x7f660410e7f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009990 tx=0x7f65fc009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 -- 172.21.15.47:0/611636652 shutdown_connections 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 --2- 172.21.15.47:0/611636652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604102300 0x7f660410e7f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 --2- 172.21.15.47:0/611636652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6604107990 0x7f6604107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 --2- 172.21.15.47:0/611636652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6604101940 0x7f6604101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 -- 172.21.15.47:0/611636652 >> 172.21.15.47:0/611636652 conn(0x7f66040788f0 msgr2=0x7f6604078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 -- 172.21.15.47:0/611636652 shutdown_connections 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.172+0000 7f660c010700 1 -- 172.21.15.47:0/611636652 wait complete. 2024-04-25T20:55:54.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660c010700 1 Processor -- start 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660c010700 1 -- start start 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660c010700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6604101940 0x7f66041992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660c010700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6604102300 0x7f6604199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660c010700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604107990 0x7f660419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660c010700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6604111c20 con 0x7f6604107990 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660c010700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6604111aa0 con 0x7f6604102300 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660c010700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6604111da0 con 0x7f6604101940 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660a5ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604107990 0x7f660419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660a5ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604107990 0x7f660419db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46154/0 (socket says 172.21.15.47:46154) 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f660a5ad700 1 -- 172.21.15.47:0/1552039710 learned_addr learned my addr 172.21.15.47:0/1552039710 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f66095ab700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6604102300 0x7f6604199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.173+0000 7f6609dac700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6604101940 0x7f66041992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:54.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f660a5ad700 1 -- 172.21.15.47:0/1552039710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6604101940 msgr2=0x7f66041992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f660a5ad700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6604101940 0x7f66041992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f660a5ad700 1 -- 172.21.15.47:0/1552039710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6604102300 msgr2=0x7f6604199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f660a5ad700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6604102300 0x7f6604199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f660a5ad700 1 -- 172.21.15.47:0/1552039710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660419e2d0 con 0x7f6604107990 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f660a5ad700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604107990 0x7f660419db90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c010 tx=0x7f65fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f6602ffd700 1 -- 172.21.15.47:0/1552039710 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc004830 con 0x7f6604107990 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660419e560 con 0x7f6604107990 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f6602ffd700 1 -- 172.21.15.47:0/1552039710 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65fc005020 con 0x7f6604107990 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f6602ffd700 1 -- 172.21.15.47:0/1552039710 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc003e90 con 0x7f6604107990 2024-04-25T20:55:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.174+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66041a5e40 con 0x7f6604107990 2024-04-25T20:55:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.175+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6604069730 con 0x7f6604107990 2024-04-25T20:55:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.176+0000 7f6602ffd700 1 -- 172.21.15.47:0/1552039710 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f65fc0051d0 con 0x7f6604107990 2024-04-25T20:55:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.177+0000 7f6602ffd700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f65f40a80d0 0x7f65f40aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.177+0000 7f6609dac700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f65f40a80d0 0x7f65f40aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.177+0000 7f6602ffd700 1 -- 172.21.15.47:0/1552039710 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f65fc0d0f10 con 0x7f6604107990 2024-04-25T20:55:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.177+0000 7f6609dac700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f65f40a80d0 0x7f65f40aa590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65f800a400 tx=0x7f65f800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.181+0000 7f6602ffd700 1 -- 172.21.15.47:0/1552039710 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65fc013070 con 0x7f6604107990 2024-04-25T20:55:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.392+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f660402df80 con 0x7f6604107990 2024-04-25T20:55:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.394+0000 7f6602ffd700 1 -- 172.21.15.47:0/1552039710 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18397 (secure 0 0 0) 0x7f65fc09dd80 con 0x7f6604107990 2024-04-25T20:55:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:55:54.395 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:55:49.276097+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f65f40a80d0 msgr2=0x7f65f40aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f65f40a80d0 0x7f65f40aa590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65f800a400 tx=0x7f65f800d040 comp rx=0 tx=0).stop 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604107990 msgr2=0x7f660419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604107990 0x7f660419db90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c010 tx=0x7f65fc0047b0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 shutdown_connections 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f65f40a80d0 0x7f65f40aa590 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6604107990 0x7f660419db90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6604101940 0x7f66041992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 --2- 172.21.15.47:0/1552039710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6604102300 0x7f6604199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 >> 172.21.15.47:0/1552039710 conn(0x7f66040788f0 msgr2=0x7f660406ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 shutdown_connections 2024-04-25T20:55:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:54.396+0000 7f660c010700 1 -- 172.21.15.47:0/1552039710 wait complete. 2024-04-25T20:55:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:54 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1552039710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:54.762 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:54 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1552039710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:54 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1552039710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:55.043 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T20:52:09.901826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T20:55:55.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T20:55:55.299 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:55.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:55 smithi092 ceph-mon[32524]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:55.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:55 smithi047 ceph-mon[29195]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:55.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:55 smithi186 ceph-mon[32529]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.936+0000 7f415367e700 1 -- 172.21.15.47:0/1482025009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 msgr2=0x7f414c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.936+0000 7f415367e700 1 --2- 172.21.15.47:0/1482025009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 0x7f414c102f10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f413c009990 tx=0x7f413c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.936+0000 7f415367e700 1 -- 172.21.15.47:0/1482025009 shutdown_connections 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.936+0000 7f415367e700 1 --2- 172.21.15.47:0/1482025009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f414c103450 0x7f414c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.936+0000 7f415367e700 1 --2- 172.21.15.47:0/1482025009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f414c108a50 0x7f414c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.936+0000 7f415367e700 1 --2- 172.21.15.47:0/1482025009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 0x7f414c102f10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.936+0000 7f415367e700 1 -- 172.21.15.47:0/1482025009 >> 172.21.15.47:0/1482025009 conn(0x7f414c079ba0 msgr2=0x7f414c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.937+0000 7f415367e700 1 -- 172.21.15.47:0/1482025009 shutdown_connections 2024-04-25T20:55:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.937+0000 7f415367e700 1 -- 172.21.15.47:0/1482025009 wait complete. 2024-04-25T20:55:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.937+0000 7f415367e700 1 Processor -- start 2024-04-25T20:55:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.937+0000 7f415367e700 1 -- start start 2024-04-25T20:55:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415367e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 0x7f414c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415367e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f414c103450 0x7f414c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415367e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f414c108a50 0x7f414c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415367e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f414c112c20 con 0x7f414c102a90 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415367e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f414c112aa0 con 0x7f414c108a50 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415367e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f414c112da0 con 0x7f414c103450 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 0x7f414c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 0x7f414c1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46166/0 (socket says 172.21.15.47:46166) 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 -- 172.21.15.47:0/343009085 learned_addr learned my addr 172.21.15.47:0/343009085 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f4151c1b700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f414c108a50 0x7f414c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f4150c19700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f414c103450 0x7f414c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 -- 172.21.15.47:0/343009085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f414c103450 msgr2=0x7f414c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f414c103450 0x7f414c199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 -- 172.21.15.47:0/343009085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f414c108a50 msgr2=0x7f414c19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f414c108a50 0x7f414c19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 -- 172.21.15.47:0/343009085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c19e1e0 con 0x7f414c102a90 2024-04-25T20:55:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.938+0000 7f415141a700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 0x7f414c1991d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f414800d980 tx=0x7f414800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.939+0000 7f41427fc700 1 -- 172.21.15.47:0/343009085 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148016070 con 0x7f414c102a90 2024-04-25T20:55:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.939+0000 7f41427fc700 1 -- 172.21.15.47:0/343009085 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4148004830 con 0x7f414c102a90 2024-04-25T20:55:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.939+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c19e4d0 con 0x7f414c102a90 2024-04-25T20:55:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.939+0000 7f41427fc700 1 -- 172.21.15.47:0/343009085 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148005540 con 0x7f414c102a90 2024-04-25T20:55:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.939+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c074c70 con 0x7f414c102a90 2024-04-25T20:55:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.940+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f414c104180 con 0x7f414c102a90 2024-04-25T20:55:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.941+0000 7f41427fc700 1 -- 172.21.15.47:0/343009085 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4148004220 con 0x7f414c102a90 2024-04-25T20:55:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.941+0000 7f41427fc700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f41380a3c40 0x7f41380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.942+0000 7f4150c19700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f41380a3c40 0x7f41380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.942+0000 7f41427fc700 1 -- 172.21.15.47:0/343009085 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f41480cfd00 con 0x7f414c102a90 2024-04-25T20:55:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.942+0000 7f4150c19700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f41380a3c40 0x7f41380a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f413c00b5c0 tx=0x7f413c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:56.946+0000 7f41427fc700 1 -- 172.21.15.47:0/343009085 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f414801f020 con 0x7f414c102a90 2024-04-25T20:55:57.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.149+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f414c19a530 con 0x7f414c102a90 2024-04-25T20:55:57.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.150+0000 7f41427fc700 1 -- 172.21.15.47:0/343009085 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f4148022350 con 0x7f414c102a90 2024-04-25T20:55:57.151 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-25T20:55:57.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.151+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f41380a3c40 msgr2=0x7f41380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:57.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.151+0000 7f415367e700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f41380a3c40 0x7f41380a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f413c00b5c0 tx=0x7f413c005960 comp rx=0 tx=0).stop 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.151+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 msgr2=0x7f414c1991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.151+0000 7f415367e700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 0x7f414c1991d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f414800d980 tx=0x7f414800dc90 comp rx=0 tx=0).stop 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.152+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 shutdown_connections 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.152+0000 7f415367e700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f41380a3c40 0x7f41380a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.152+0000 7f415367e700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c102a90 0x7f414c1991d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.152+0000 7f415367e700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f414c108a50 0x7f414c19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.152+0000 7f415367e700 1 --2- 172.21.15.47:0/343009085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f414c103450 0x7f414c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.152+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 >> 172.21.15.47:0/343009085 conn(0x7f414c079ba0 msgr2=0x7f414c1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.152+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 shutdown_connections 2024-04-25T20:55:57.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:55:57.152+0000 7f415367e700 1 -- 172.21.15.47:0/343009085 wait complete. 2024-04-25T20:55:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:57 smithi186 ceph-mon[32529]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:57 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/343009085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T20:55:57.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:57 smithi047 ceph-mon[29195]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:57.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:57 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/343009085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T20:55:57.749 INFO:tasks.cephadm:Adding mds.a on smithi047 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.d on smithi047 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.g on smithi047 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.j on smithi047 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.b on smithi092 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.e on smithi092 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.h on smithi092 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.k on smithi092 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.c on smithi186 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.f on smithi186 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.i on smithi186 2024-04-25T20:55:57.750 INFO:tasks.cephadm:Adding mds.l on smithi186 2024-04-25T20:55:57.751 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2024-04-25T20:55:57.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:57 smithi092 ceph-mon[32524]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:57.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:57 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/343009085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T20:55:58.002 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:55:59.545 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:55:59 smithi186 ceph-mon[32529]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:59.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1565587107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0103ca0 msgr2=0x7fa3c0104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:59.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1565587107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0103ca0 0x7fa3c0104100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009990 tx=0x7fa3ac009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1565587107 shutdown_connections 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1565587107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c0104640 0x7fa3c010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1565587107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3c0102aa0 0x7fa3c0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1565587107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0103ca0 0x7fa3c0104100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1565587107 >> 172.21.15.186:0/1565587107 conn(0x7fa3c00fe040 msgr2=0x7fa3c0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1565587107 shutdown_connections 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1565587107 wait complete. 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 Processor -- start 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- start start 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0102aa0 0x7fa3c019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3c0103ca0 0x7fa3c019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c0104640 0x7fa3c01a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010e630 con 0x7fa3c0104640 2024-04-25T20:55:59.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010e4b0 con 0x7fa3c0103ca0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.772+0000 7fa3c6f5a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010e7b0 con 0x7fa3c0102aa0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0102aa0 0x7fa3c019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0102aa0 0x7fa3c019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:54168/0 (socket says 172.21.15.186:54168) 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 -- 172.21.15.186:0/1783808258 learned_addr learned my addr 172.21.15.186:0/1783808258 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3bffff700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3c0103ca0 0x7fa3c019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c54f7700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c0104640 0x7fa3c01a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 -- 172.21.15.186:0/1783808258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3c0103ca0 msgr2=0x7fa3c019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3c0103ca0 0x7fa3c019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 -- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c0104640 msgr2=0x7fa3c01a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c0104640 0x7fa3c01a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 -- 172.21.15.186:0/1783808258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c01a2b10 con 0x7fa3c0102aa0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c4cf6700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0102aa0 0x7fa3c019aeb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400b530 tx=0x7fa3b400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3bdffb700 1 -- 172.21.15.186:0/1783808258 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4015020 con 0x7fa3c0102aa0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3bdffb700 1 -- 172.21.15.186:0/1783808258 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b4011ce0 con 0x7fa3c0102aa0 2024-04-25T20:55:59.776 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c01a2e00 con 0x7fa3c0102aa0 2024-04-25T20:55:59.777 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3bdffb700 1 -- 172.21.15.186:0/1783808258 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4010d60 con 0x7fa3c0102aa0 2024-04-25T20:55:59.777 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.773+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01a3270 con 0x7fa3c0102aa0 2024-04-25T20:55:59.777 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.774+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c0069760 con 0x7fa3c0102aa0 2024-04-25T20:55:59.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.775+0000 7fa3bdffb700 1 -- 172.21.15.186:0/1783808258 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa3b4018020 con 0x7fa3c0102aa0 2024-04-25T20:55:59.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.776+0000 7fa3bdffb700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa3b00a3c40 0x7fa3b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:59.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.776+0000 7fa3bdffb700 1 -- 172.21.15.186:0/1783808258 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa3b40cf6f0 con 0x7fa3c0102aa0 2024-04-25T20:55:59.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.776+0000 7fa3bffff700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa3b00a3c40 0x7fa3b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:59.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.777+0000 7fa3bffff700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa3b00a3c40 0x7fa3b00a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b5c0 tx=0x7fa3ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:59.781 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.781+0000 7fa3bdffb700 1 -- 172.21.15.186:0/1783808258 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3c0069760 con 0x7fa3c0102aa0 2024-04-25T20:55:59.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:55:59 smithi092 ceph-mon[32524]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:59.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:55:59 smithi047 ceph-mon[29195]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:55:59.990 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.989+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) -- 0x7fa3c0076490 con 0x7fa3b00a3c40 2024-04-25T20:55:59.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.997+0000 7fa3bdffb700 1 -- 172.21.15.186:0/1783808258 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fa3c0076490 con 0x7fa3b00a3c40 2024-04-25T20:55:59.998 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2024-04-25T20:56:00.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.999+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa3b00a3c40 msgr2=0x7fa3b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.999+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa3b00a3c40 0x7fa3b00a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b5c0 tx=0x7fa3ac005fb0 comp rx=0 tx=0).stop 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.999+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0102aa0 msgr2=0x7fa3c019aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:55:59.999+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0102aa0 0x7fa3c019aeb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400b530 tx=0x7fa3b400b840 comp rx=0 tx=0).stop 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:56:00.000+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 shutdown_connections 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:56:00.000+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa3b00a3c40 0x7fa3b00a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:56:00.000+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c0104640 0x7fa3c01a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:56:00.000+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3c0102aa0 0x7fa3c019aeb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:56:00.000+0000 7fa3c6f5a700 1 --2- 172.21.15.186:0/1783808258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3c0103ca0 0x7fa3c019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:56:00.000+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 >> 172.21.15.186:0/1783808258 conn(0x7fa3c00fe040 msgr2=0x7fa3c0074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:56:00.000+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 shutdown_connections 2024-04-25T20:56:00.001 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T20:56:00.000+0000 7fa3c6f5a700 1 -- 172.21.15.186:0/1783808258 wait complete. 2024-04-25T20:56:00.496 DEBUG:teuthology.orchestra.run.smithi047:mds.a> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T20:56:00.499 DEBUG:teuthology.orchestra.run.smithi047:mds.d> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T20:56:00.543 DEBUG:teuthology.orchestra.run.smithi047:mds.g> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T20:56:00.586 DEBUG:teuthology.orchestra.run.smithi047:mds.j> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T20:56:00.629 DEBUG:teuthology.orchestra.run.smithi092:mds.b> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T20:56:00.632 DEBUG:teuthology.orchestra.run.smithi092:mds.e> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T20:56:00.674 DEBUG:teuthology.orchestra.run.smithi092:mds.h> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T20:56:00.717 DEBUG:teuthology.orchestra.run.smithi092:mds.k> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T20:56:00.760 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T20:56:00.762 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T20:56:00.805 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T20:56:00.848 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T20:56:00.891 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T20:56:00.892 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:01.138 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:01.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:00 smithi186 ceph-mon[32529]: Deploying daemon mds.l on smithi186 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:01.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:01.249 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:00 smithi047 ceph-mon[29195]: Deploying daemon mds.l on smithi186 2024-04-25T20:56:01.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:01.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:00 smithi092 ceph-mon[32524]: Deploying daemon mds.l on smithi186 2024-04-25T20:56:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 -- 172.21.15.47:0/3928379257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054102300 msgr2=0x7f505410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 --2- 172.21.15.47:0/3928379257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054102300 0x7f505410e7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f504c009990 tx=0x7f504c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 -- 172.21.15.47:0/3928379257 shutdown_connections 2024-04-25T20:56:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 --2- 172.21.15.47:0/3928379257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054102300 0x7f505410e7f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 --2- 172.21.15.47:0/3928379257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5054107990 0x7f5054107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 --2- 172.21.15.47:0/3928379257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5054101940 0x7f5054101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 -- 172.21.15.47:0/3928379257 >> 172.21.15.47:0/3928379257 conn(0x7f50540788f0 msgr2=0x7f5054078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 -- 172.21.15.47:0/3928379257 shutdown_connections 2024-04-25T20:56:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.701+0000 7f50588ac700 1 -- 172.21.15.47:0/3928379257 wait complete. 2024-04-25T20:56:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f50588ac700 1 Processor -- start 2024-04-25T20:56:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f50588ac700 1 -- start start 2024-04-25T20:56:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f50588ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5054101940 0x7f5054197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f50588ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5054102300 0x7f5054198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f50588ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054107990 0x7f50541987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f50588ac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5054111ae0 con 0x7f5054107990 2024-04-25T20:56:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f50588ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5054111960 con 0x7f5054101940 2024-04-25T20:56:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f50588ac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5054111c60 con 0x7f5054102300 2024-04-25T20:56:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5052d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054107990 0x7f50541987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5052d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054107990 0x7f50541987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46184/0 (socket says 172.21.15.47:46184) 2024-04-25T20:56:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5052d9d700 1 -- 172.21.15.47:0/4129512860 learned_addr learned my addr 172.21.15.47:0/4129512860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5051d9b700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5054102300 0x7f5054198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f505259c700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5054101940 0x7f5054197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5052d9d700 1 -- 172.21.15.47:0/4129512860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5054102300 msgr2=0x7f5054198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5052d9d700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5054102300 0x7f5054198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5052d9d700 1 -- 172.21.15.47:0/4129512860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5054101940 msgr2=0x7f5054197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5052d9d700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5054101940 0x7f5054197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.702+0000 7f5052d9d700 1 -- 172.21.15.47:0/4129512860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50541a75d0 con 0x7f5054107990 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.703+0000 7f5052d9d700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054107990 0x7f50541987d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f504c0057c0 tx=0x7f504c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.703+0000 7f50437fe700 1 -- 172.21.15.47:0/4129512860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c027070 con 0x7f5054107990 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.703+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50541a7860 con 0x7f5054107990 2024-04-25T20:56:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.703+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50541a7cb0 con 0x7f5054107990 2024-04-25T20:56:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.703+0000 7f50437fe700 1 -- 172.21.15.47:0/4129512860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f504c003680 con 0x7f5054107990 2024-04-25T20:56:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.703+0000 7f50437fe700 1 -- 172.21.15.47:0/4129512860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c022b40 con 0x7f5054107990 2024-04-25T20:56:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.704+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5034005330 con 0x7f5054107990 2024-04-25T20:56:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.705+0000 7f50437fe700 1 -- 172.21.15.47:0/4129512860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f504c02a070 con 0x7f5054107990 2024-04-25T20:56:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.706+0000 7f50437fe700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f503c0a3cc0 0x7f503c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.706+0000 7f505259c700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f503c0a3cc0 0x7f503c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.706+0000 7f50437fe700 1 -- 172.21.15.47:0/4129512860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f504c0d0f60 con 0x7f5054107990 2024-04-25T20:56:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.706+0000 7f505259c700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f503c0a3cc0 0x7f503c0a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5044009550 tx=0x7f50440091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.709+0000 7f50437fe700 1 -- 172.21.15.47:0/4129512860 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f504c031080 con 0x7f5054107990 2024-04-25T20:56:02.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.963+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5034005620 con 0x7f5054107990 2024-04-25T20:56:02.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.964+0000 7f50437fe700 1 -- 172.21.15.47:0/4129512860 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f504c013070 con 0x7f5054107990 2024-04-25T20:56:02.966 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:02.966 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T20:56:02.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f503c0a3cc0 msgr2=0x7f503c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:02.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f503c0a3cc0 0x7f503c0a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5044009550 tx=0x7f50440091f0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054107990 msgr2=0x7f50541987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:02.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054107990 0x7f50541987d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f504c0057c0 tx=0x7f504c0057f0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 shutdown_connections 2024-04-25T20:56:02.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f503c0a3cc0 0x7f503c0a6180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5054107990 0x7f50541987d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5054101940 0x7f5054197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 --2- 172.21.15.47:0/4129512860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5054102300 0x7f5054198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:02.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 >> 172.21.15.47:0/4129512860 conn(0x7f50540788f0 msgr2=0x7f50540ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:02.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 shutdown_connections 2024-04-25T20:56:02.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:02.966+0000 7f50588ac700 1 -- 172.21.15.47:0/4129512860 wait complete. 2024-04-25T20:56:02.969 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 1 2024-04-25T20:56:02.999 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 20:56:02 smithi186 podman[61813]: 2024-04-25 20:56:02.967785485 +0000 UTC m=+0.116548079 container create de88cee1ac37004e0da2563989d47b5d0f6a9b6e3f37d3e93df1d6f3df390143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T20:56:03.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:02 smithi092 ceph-mon[32524]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:03.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:02 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/4129512860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:03.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:02 smithi047 ceph-mon[29195]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:03.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:02 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/4129512860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:03.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:02 smithi186 ceph-mon[32529]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:03.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:02 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/4129512860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:03.471 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 20:56:03 smithi186 podman[61813]: 2024-04-25 20:56:03.153591493 +0000 UTC m=+0.302354087 container init de88cee1ac37004e0da2563989d47b5d0f6a9b6e3f37d3e93df1d6f3df390143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T20:56:03.471 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 20:56:03 smithi186 podman[61813]: 2024-04-25 20:56:03.15631425 +0000 UTC m=+0.305076853 container start de88cee1ac37004e0da2563989d47b5d0f6a9b6e3f37d3e93df1d6f3df390143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T20:56:03.471 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 20:56:03 smithi186 ceph-mds[61832]: -- 172.21.15.186:0/98227348 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4b4899a40 con 0x55a4b486d000 2024-04-25T20:56:03.471 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 20:56:03 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l[61828]: starting mds.l at 2024-04-25T20:56:03.471 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 20:56:03 smithi186 bash[61813]: de88cee1ac37004e0da2563989d47b5d0f6a9b6e3f37d3e93df1d6f3df390143 2024-04-25T20:56:03.472 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 20:56:03 smithi186 systemd[1]: Started Ceph mds.l for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.186:6832/3007060622,v1:172.21.15.186:6833/3007060622] up:boot 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: fsmap 1 up:standby 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:04 smithi092 ceph-mon[32524]: Deploying daemon mds.k on smithi092 2024-04-25T20:56:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.186:6832/3007060622,v1:172.21.15.186:6833/3007060622] up:boot 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: fsmap 1 up:standby 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:04 smithi186 ceph-mon[32529]: Deploying daemon mds.k on smithi092 2024-04-25T20:56:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.186:6832/3007060622,v1:172.21.15.186:6833/3007060622] up:boot 2024-04-25T20:56:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: fsmap 1 up:standby 2024-04-25T20:56:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:56:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:04 smithi047 ceph-mon[29195]: Deploying daemon mds.k on smithi092 2024-04-25T20:56:05.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:05 smithi047 ceph-mon[29195]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:05.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:05.694 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:05 smithi092 ceph-mon[32524]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:05.694 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:05 smithi092 systemd[1]: Starting Ceph mds.k for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:56:05.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:05 smithi186 ceph-mon[32529]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:05.886 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:06.022 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:05 smithi092 podman[61858]: 2024-04-25 20:56:05.693840185 +0000 UTC m=+0.107837839 container create 76869c1f7439d2ead7479878893fe7c6e25ebd1d290faee6c3eddbb8dd0be165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T20:56:06.022 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:05 smithi092 podman[61858]: 2024-04-25 20:56:05.603878222 +0000 UTC m=+0.017875890 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:56:06.022 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:05 smithi092 podman[61858]: 2024-04-25 20:56:05.904768371 +0000 UTC m=+0.318766025 container init 76869c1f7439d2ead7479878893fe7c6e25ebd1d290faee6c3eddbb8dd0be165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T20:56:06.022 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:05 smithi092 podman[61858]: 2024-04-25 20:56:05.907306964 +0000 UTC m=+0.321304623 container start 76869c1f7439d2ead7479878893fe7c6e25ebd1d290faee6c3eddbb8dd0be165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:56:06.022 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:05 smithi092 ceph-mds[61877]: -- 172.21.15.92:0/631396158 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f249845a40 con 0x55f249818800 2024-04-25T20:56:06.022 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k[61873]: starting mds.k at 2024-04-25T20:56:06.022 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:06 smithi092 bash[61858]: 76869c1f7439d2ead7479878893fe7c6e25ebd1d290faee6c3eddbb8dd0be165 2024-04-25T20:56:06.282 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 20:56:06 smithi092 systemd[1]: Started Ceph mds.k for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:07.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:07.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:07.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:07.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:07.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: Deploying daemon mds.j on smithi047 2024-04-25T20:56:07.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6832/1380167956,v1:172.21.15.92:6833/1380167956] up:boot 2024-04-25T20:56:07.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: fsmap 2 up:standby 2024-04-25T20:56:07.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:07 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:56:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: Deploying daemon mds.j on smithi047 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6832/1380167956,v1:172.21.15.92:6833/1380167956] up:boot 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: fsmap 2 up:standby 2024-04-25T20:56:07.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:07 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:56:07.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3953008043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 msgr2=0x7f4cc006bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3953008043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 0x7f4cc006bb30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009990 tx=0x7f4cb4009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3953008043 shutdown_connections 2024-04-25T20:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3953008043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc0105470 0x7f4cc0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3953008043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0107960 0x7f4cc006b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3953008043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 0x7f4cc006bb30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3953008043 >> 172.21.15.47:0/3953008043 conn(0x7f4cc00788f0 msgr2=0x7f4cc0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3953008043 shutdown_connections 2024-04-25T20:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3953008043 wait complete. 2024-04-25T20:56:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.484+0000 7f4cc5a20700 1 Processor -- start 2024-04-25T20:56:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cc5a20700 1 -- start start 2024-04-25T20:56:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cc5a20700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 0x7f4cc019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cc5a20700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc0105470 0x7f4cc019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cc5a20700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0107960 0x7f4cc01a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cc5a20700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01191c0 con 0x7f4cc006b6b0 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cc5a20700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0119040 con 0x7f4cc0105470 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cc5a20700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0119340 con 0x7f4cc0107960 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 0x7f4cc019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 0x7f4cc019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33628/0 (socket says 172.21.15.47:33628) 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 -- 172.21.15.47:0/3133956292 learned_addr learned my addr 172.21.15.47:0/3133956292 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbe7fc700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc0105470 0x7f4cc019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbf7fe700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0107960 0x7f4cc01a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 -- 172.21.15.47:0/3133956292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0107960 msgr2=0x7f4cc01a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0107960 0x7f4cc01a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 -- 172.21.15.47:0/3133956292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc0105470 msgr2=0x7f4cc019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc0105470 0x7f4cc019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 -- 172.21.15.47:0/3133956292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc01a46e0 con 0x7f4cc006b6b0 2024-04-25T20:56:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.485+0000 7f4cbeffd700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 0x7f4cc019f6d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000d950 tx=0x7f4cb000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.486+0000 7f4cc4a1e700 1 -- 172.21.15.47:0/3133956292 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0016070 con 0x7f4cc006b6b0 2024-04-25T20:56:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.486+0000 7f4cc4a1e700 1 -- 172.21.15.47:0/3133956292 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb0009df0 con 0x7f4cc006b6b0 2024-04-25T20:56:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.486+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc01a49d0 con 0x7f4cc006b6b0 2024-04-25T20:56:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.486+0000 7f4cc4a1e700 1 -- 172.21.15.47:0/3133956292 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000f3c0 con 0x7f4cc006b6b0 2024-04-25T20:56:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.486+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc01ac250 con 0x7f4cc006b6b0 2024-04-25T20:56:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.487+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc0101f50 con 0x7f4cc006b6b0 2024-04-25T20:56:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.488+0000 7f4cc4a1e700 1 -- 172.21.15.47:0/3133956292 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4cb0019070 con 0x7f4cc006b6b0 2024-04-25T20:56:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.488+0000 7f4cc4a1e700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4ca80a3c40 0x7f4ca80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.488+0000 7f4cbe7fc700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4ca80a3c40 0x7f4ca80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.489+0000 7f4cc4a1e700 1 -- 172.21.15.47:0/3133956292 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4cb00cfa90 con 0x7f4cc006b6b0 2024-04-25T20:56:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.489+0000 7f4cbe7fc700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4ca80a3c40 0x7f4ca80a6100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b5c0 tx=0x7f4cb4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:07.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.495+0000 7f4cc4a1e700 1 -- 172.21.15.47:0/3133956292 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cb009c910 con 0x7f4cc006b6b0 2024-04-25T20:56:07.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:07.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:07.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:07.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:07.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:07.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: Deploying daemon mds.j on smithi047 2024-04-25T20:56:07.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6832/1380167956,v1:172.21.15.92:6833/1380167956] up:boot 2024-04-25T20:56:07.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: fsmap 2 up:standby 2024-04-25T20:56:07.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:07 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:56:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.777+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cc0069730 con 0x7f4cc006b6b0 2024-04-25T20:56:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.778+0000 7f4cc4a1e700 1 -- 172.21.15.47:0/3133956292 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f4cb009c210 con 0x7f4cc006b6b0 2024-04-25T20:56:07.779 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:07.779 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4ca80a3c40 msgr2=0x7f4ca80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4ca80a3c40 0x7f4ca80a6100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b5c0 tx=0x7f4cb4005960 comp rx=0 tx=0).stop 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 msgr2=0x7f4cc019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 0x7f4cc019f6d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000d950 tx=0x7f4cb000dc60 comp rx=0 tx=0).stop 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 shutdown_connections 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4ca80a3c40 0x7f4ca80a6100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc006b6b0 0x7f4cc019f6d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0107960 0x7f4cc01a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 --2- 172.21.15.47:0/3133956292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc0105470 0x7f4cc019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 >> 172.21.15.47:0/3133956292 conn(0x7f4cc00788f0 msgr2=0x7f4cc0104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:07.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 shutdown_connections 2024-04-25T20:56:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:07.780+0000 7f4cc5a20700 1 -- 172.21.15.47:0/3133956292 wait complete. 2024-04-25T20:56:07.782 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-04-25T20:56:08.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:08 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3133956292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:08.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:08 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3133956292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:08.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:08 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3133956292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:09.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:09 smithi047 ceph-mon[29195]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:09.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:09 smithi186 ceph-mon[32529]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:09.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:09 smithi092 ceph-mon[32524]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:10.111 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 20:56:10 smithi047 podman[67224]: 2024-04-25 20:56:10.003170232 +0000 UTC m=+0.125342331 container create 594afccd3cb2e4339c31b53aeecc6c94b155a460441b3856386aeb2f305797cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T20:56:10.611 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 20:56:10 smithi047 podman[67224]: 2024-04-25 20:56:10.247968978 +0000 UTC m=+0.370141069 container init 594afccd3cb2e4339c31b53aeecc6c94b155a460441b3856386aeb2f305797cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T20:56:10.611 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 20:56:10 smithi047 podman[67224]: 2024-04-25 20:56:10.250615186 +0000 UTC m=+0.372787275 container start 594afccd3cb2e4339c31b53aeecc6c94b155a460441b3856386aeb2f305797cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T20:56:10.611 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 20:56:10 smithi047 ceph-mds[67243]: -- 172.21.15.47:0/1753796945 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b957181a40 con 0x55b957154800 2024-04-25T20:56:10.611 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 20:56:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j[67239]: starting mds.j at 2024-04-25T20:56:10.611 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 20:56:10 smithi047 bash[67224]: 594afccd3cb2e4339c31b53aeecc6c94b155a460441b3856386aeb2f305797cc 2024-04-25T20:56:10.611 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 20:56:10 smithi047 systemd[1]: Started Ceph mds.j for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:10.810 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: Deploying daemon mds.d on smithi047 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.47:6834/911765730,v1:172.21.15.47:6835/911765730] up:boot 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: fsmap 3 up:standby 2024-04-25T20:56:11.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:11 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:56:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: Deploying daemon mds.d on smithi047 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.47:6834/911765730,v1:172.21.15.47:6835/911765730] up:boot 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: fsmap 3 up:standby 2024-04-25T20:56:11.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:11 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:56:11.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: Deploying daemon mds.d on smithi047 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.47:6834/911765730,v1:172.21.15.47:6835/911765730] up:boot 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: fsmap 3 up:standby 2024-04-25T20:56:11.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:11 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:56:12.612 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:12 smithi047 systemd[1]: Starting Ceph mds.d for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:56:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/2562971200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 msgr2=0x7f7d9810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/2562971200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 0x7f7d9810e7f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000b0a0 tx=0x7f7d9000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/2562971200 shutdown_connections 2024-04-25T20:56:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/2562971200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 0x7f7d9810e7f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/2562971200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98107990 0x7f7d98107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/2562971200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d98101940 0x7f7d98101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/2562971200 >> 172.21.15.47:0/2562971200 conn(0x7f7d980788f0 msgr2=0x7f7d98078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/2562971200 shutdown_connections 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.513+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/2562971200 wait complete. 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d9e9cf700 1 Processor -- start 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d9e9cf700 1 -- start start 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d9e9cf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d98101940 0x7f7d98199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d9e9cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 0x7f7d9819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d9e9cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98107990 0x7f7d981a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d9e9cf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98111ae0 con 0x7f7d98102300 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d9e9cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98111960 con 0x7f7d98107990 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d9e9cf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98111c60 con 0x7f7d98101940 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.514+0000 7f7d977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 0x7f7d9819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 0x7f7d9819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33668/0 (socket says 172.21.15.47:33668) 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d977fe700 1 -- 172.21.15.47:0/3954334093 learned_addr learned my addr 172.21.15.47:0/3954334093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d9cf6c700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98107990 0x7f7d981a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d97fff700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d98101940 0x7f7d98199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d977fe700 1 -- 172.21.15.47:0/3954334093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d98101940 msgr2=0x7f7d98199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d977fe700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d98101940 0x7f7d98199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d977fe700 1 -- 172.21.15.47:0/3954334093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98107990 msgr2=0x7f7d981a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d977fe700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98107990 0x7f7d981a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d977fe700 1 -- 172.21.15.47:0/3954334093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d981a5560 con 0x7f7d98102300 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d97fff700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d98101940 0x7f7d98199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d977fe700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 0x7f7d9819a4e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00c9b0 tx=0x7f7d8c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d957fa700 1 -- 172.21.15.47:0/3954334093 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c007b00 con 0x7f7d98102300 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d981a5730 con 0x7f7d98102300 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d957fa700 1 -- 172.21.15.47:0/3954334093 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c007ca0 con 0x7f7d98102300 2024-04-25T20:56:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d957fa700 1 -- 172.21.15.47:0/3954334093 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c018660 con 0x7f7d98102300 2024-04-25T20:56:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.515+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d981a5c40 con 0x7f7d98102300 2024-04-25T20:56:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.516+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d98103060 con 0x7f7d98102300 2024-04-25T20:56:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.517+0000 7f7d957fa700 1 -- 172.21.15.47:0/3954334093 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7d8c00f450 con 0x7f7d98102300 2024-04-25T20:56:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.518+0000 7f7d957fa700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7d800a3c40 0x7f7d800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.518+0000 7f7d97fff700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7d800a3c40 0x7f7d800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.518+0000 7f7d957fa700 1 -- 172.21.15.47:0/3954334093 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7d8c0cfff0 con 0x7f7d98102300 2024-04-25T20:56:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.518+0000 7f7d97fff700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7d800a3c40 0x7f7d800a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800bb90 tx=0x7f7d88005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.523+0000 7f7d957fa700 1 -- 172.21.15.47:0/3954334093 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d8c014030 con 0x7f7d98102300 2024-04-25T20:56:13.593 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:13 smithi047 ceph-mon[29195]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:13.594 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:13 smithi047 podman[67643]: 2024-04-25 20:56:13.319325087 +0000 UTC m=+0.023081284 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:56:13.594 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:13 smithi047 podman[67643]: 2024-04-25 20:56:13.593576138 +0000 UTC m=+0.297332324 container create 279f0681eb37e63964968a7887cacd97b5db9fd25967dddedd0050f4cf8a4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T20:56:13.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:13 smithi186 ceph-mon[32529]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:13.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:13 smithi092 ceph-mon[32524]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:13.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.796+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d9806acc0 con 0x7f7d98102300 2024-04-25T20:56:13.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.797+0000 7f7d957fa700 1 -- 172.21.15.47:0/3954334093 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f7d8c09cdf0 con 0x7f7d98102300 2024-04-25T20:56:13.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:13.798 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7d800a3c40 msgr2=0x7f7d800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7d800a3c40 0x7f7d800a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800bb90 tx=0x7f7d88005d80 comp rx=0 tx=0).stop 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 msgr2=0x7f7d9819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 0x7f7d9819a4e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00c9b0 tx=0x7f7d8c00ccc0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 shutdown_connections 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7d800a3c40 0x7f7d800a6100 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98107990 0x7f7d981a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d98101940 0x7f7d98199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 --2- 172.21.15.47:0/3954334093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d98102300 0x7f7d9819a4e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:13.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 >> 172.21.15.47:0/3954334093 conn(0x7f7d980788f0 msgr2=0x7f7d980ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:13.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 shutdown_connections 2024-04-25T20:56:13.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:13.799+0000 7f7d9e9cf700 1 -- 172.21.15.47:0/3954334093 wait complete. 2024-04-25T20:56:13.801 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 4 2024-04-25T20:56:13.861 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:13 smithi047 podman[67643]: 2024-04-25 20:56:13.839156379 +0000 UTC m=+0.542912565 container init 279f0681eb37e63964968a7887cacd97b5db9fd25967dddedd0050f4cf8a4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T20:56:13.861 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:13 smithi047 podman[67643]: 2024-04-25 20:56:13.842329716 +0000 UTC m=+0.546085902 container start 279f0681eb37e63964968a7887cacd97b5db9fd25967dddedd0050f4cf8a4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T20:56:14.210 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:13 smithi047 ceph-mds[67700]: -- 172.21.15.47:0/37322884 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b0446fa40 con 0x558b04442c00 2024-04-25T20:56:14.210 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:13 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d[67683]: starting mds.d at 2024-04-25T20:56:14.210 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:13 smithi047 bash[67643]: 279f0681eb37e63964968a7887cacd97b5db9fd25967dddedd0050f4cf8a4efd 2024-04-25T20:56:14.210 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 20:56:13 smithi047 systemd[1]: Started Ceph mds.d for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:14 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3954334093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:14 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:14 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:14 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:14 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:14.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:14 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:14.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:14 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:14.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:14 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3954334093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:14.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:14 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:14 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:14 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:14 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:14.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:14 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:14.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:14 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:14 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3954334093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:14 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:14 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:14 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:14.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:14 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:14.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:14 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:14.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:14 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:15.691 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:15 smithi092 ceph-mon[32524]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:15.692 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:15 smithi092 ceph-mon[32524]: Deploying daemon mds.e on smithi092 2024-04-25T20:56:15.692 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:15 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.47:6836/12948729,v1:172.21.15.47:6837/12948729] up:boot 2024-04-25T20:56:15.692 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:15 smithi092 ceph-mon[32524]: fsmap 4 up:standby 2024-04-25T20:56:15.692 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:15 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:56:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:15 smithi186 ceph-mon[32529]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:15 smithi186 ceph-mon[32529]: Deploying daemon mds.e on smithi092 2024-04-25T20:56:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:15 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.47:6836/12948729,v1:172.21.15.47:6837/12948729] up:boot 2024-04-25T20:56:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:15 smithi186 ceph-mon[32529]: fsmap 4 up:standby 2024-04-25T20:56:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:15 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:56:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:15 smithi047 ceph-mon[29195]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:15 smithi047 ceph-mon[29195]: Deploying daemon mds.e on smithi092 2024-04-25T20:56:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:15 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.47:6836/12948729,v1:172.21.15.47:6837/12948729] up:boot 2024-04-25T20:56:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:15 smithi047 ceph-mon[29195]: fsmap 4 up:standby 2024-04-25T20:56:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:15 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:56:16.590 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:17.032 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 20:56:16 smithi092 systemd[1]: Started Ceph mds.e for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:17.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] up:boot 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: fsmap 5 up:standby 2024-04-25T20:56:17.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:17 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:56:17.986 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:17.986 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:17.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:17.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:17.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:17.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:17.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:17.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] up:boot 2024-04-25T20:56:17.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: fsmap 5 up:standby 2024-04-25T20:56:17.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:17 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:56:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] up:boot 2024-04-25T20:56:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: fsmap 5 up:standby 2024-04-25T20:56:18.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:17 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:56:18.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 -- 172.21.15.47:0/3636820020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc103480 msgr2=0x7fc2dc10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:18.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 --2- 172.21.15.47:0/3636820020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc103480 0x7fc2dc10f980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400b0a0 tx=0x7fc2d400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 -- 172.21.15.47:0/3636820020 shutdown_connections 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 --2- 172.21.15.47:0/3636820020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc103480 0x7fc2dc10f980 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 --2- 172.21.15.47:0/3636820020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2dc108ab0 0x7fc2dc108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 --2- 172.21.15.47:0/3636820020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2dc102ac0 0x7fc2dc102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 -- 172.21.15.47:0/3636820020 >> 172.21.15.47:0/3636820020 conn(0x7fc2dc079bf0 msgr2=0x7fc2dc07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 -- 172.21.15.47:0/3636820020 shutdown_connections 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.512+0000 7fc2e1742700 1 -- 172.21.15.47:0/3636820020 wait complete. 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.513+0000 7fc2e1742700 1 Processor -- start 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.513+0000 7fc2e1742700 1 -- start start 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.513+0000 7fc2e1742700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2dc102ac0 0x7fc2dc199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.513+0000 7fc2e1742700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2dc103480 0x7fc2dc1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.513+0000 7fc2e1742700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc108ab0 0x7fc2dc19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.513+0000 7fc2e1742700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc112cf0 con 0x7fc2dc108ab0 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.513+0000 7fc2e1742700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc112b70 con 0x7fc2dc103480 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.513+0000 7fc2e1742700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc112e70 con 0x7fc2dc102ac0 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc108ab0 0x7fc2dc19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2daffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2dc102ac0 0x7fc2dc199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc108ab0 0x7fc2dc19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57080/0 (socket says 172.21.15.47:57080) 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 -- 172.21.15.47:0/1477919577 learned_addr learned my addr 172.21.15.47:0/1477919577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2da7fc700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2dc103480 0x7fc2dc1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 -- 172.21.15.47:0/1477919577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2dc102ac0 msgr2=0x7fc2dc199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2dc102ac0 0x7fc2dc199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 -- 172.21.15.47:0/1477919577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2dc103480 msgr2=0x7fc2dc1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2dc103480 0x7fc2dc1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 -- 172.21.15.47:0/1477919577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc19e090 con 0x7fc2dc108ab0 2024-04-25T20:56:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2daffd700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2dc102ac0 0x7fc2dc199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:56:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2db7fe700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc108ab0 0x7fc2dc19d950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40062a0 tx=0x7fc2d4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2c3fff700 1 -- 172.21.15.47:0/1477919577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4021070 con 0x7fc2dc108ab0 2024-04-25T20:56:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2c3fff700 1 -- 172.21.15.47:0/1477919577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d4007e30 con 0x7fc2dc108ab0 2024-04-25T20:56:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2dc19e320 con 0x7fc2dc108ab0 2024-04-25T20:56:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2c3fff700 1 -- 172.21.15.47:0/1477919577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40046d0 con 0x7fc2dc108ab0 2024-04-25T20:56:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.514+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2dc1a5c00 con 0x7fc2dc108ab0 2024-04-25T20:56:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.515+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2dc069730 con 0x7fc2dc108ab0 2024-04-25T20:56:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.517+0000 7fc2c3fff700 1 -- 172.21.15.47:0/1477919577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc2d4007980 con 0x7fc2dc108ab0 2024-04-25T20:56:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.517+0000 7fc2c3fff700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc2c40a3c40 0x7fc2c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.517+0000 7fc2daffd700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc2c40a3c40 0x7fc2c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.517+0000 7fc2c3fff700 1 -- 172.21.15.47:0/1477919577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc2d400f020 con 0x7fc2dc108ab0 2024-04-25T20:56:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.518+0000 7fc2daffd700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc2c40a3c40 0x7fc2c40a6100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009fd0 tx=0x7fc2d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.521+0000 7fc2c3fff700 1 -- 172.21.15.47:0/1477919577 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2d402a020 con 0x7fc2dc108ab0 2024-04-25T20:56:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.782+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2dc02df80 con 0x7fc2dc108ab0 2024-04-25T20:56:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.782+0000 7fc2c3fff700 1 -- 172.21.15.47:0/1477919577 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7fc2d409e670 con 0x7fc2dc108ab0 2024-04-25T20:56:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:18.784 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc2c40a3c40 msgr2=0x7fc2c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc2c40a3c40 0x7fc2c40a6100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009fd0 tx=0x7fc2d0009260 comp rx=0 tx=0).stop 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc108ab0 msgr2=0x7fc2dc19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc108ab0 0x7fc2dc19d950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40062a0 tx=0x7fc2d4004c80 comp rx=0 tx=0).stop 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 shutdown_connections 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fc2c40a3c40 0x7fc2c40a6100 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2dc108ab0 0x7fc2dc19d950 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2dc102ac0 0x7fc2dc199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 --2- 172.21.15.47:0/1477919577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2dc103480 0x7fc2dc1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 >> 172.21.15.47:0/1477919577 conn(0x7fc2dc079bf0 msgr2=0x7fc2dc100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 shutdown_connections 2024-04-25T20:56:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:18.785+0000 7fc2e1742700 1 -- 172.21.15.47:0/1477919577 wait complete. 2024-04-25T20:56:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-04-25T20:56:18.963 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:18 smithi186 ceph-mon[32529]: Deploying daemon mds.f on smithi186 2024-04-25T20:56:18.963 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:18 smithi186 systemd[1]: Starting Ceph mds.f for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:56:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:18 smithi092 ceph-mon[32524]: Deploying daemon mds.f on smithi186 2024-04-25T20:56:19.085 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:18 smithi047 ceph-mon[29195]: Deploying daemon mds.f on smithi186 2024-04-25T20:56:19.221 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:18 smithi186 podman[62126]: 2024-04-25 20:56:18.865368038 +0000 UTC m=+0.017952487 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:56:19.221 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:18 smithi186 podman[62126]: 2024-04-25 20:56:18.977515338 +0000 UTC m=+0.130099783 container create dd9898b1495e4f8d9de885bcffdca69bc8c5c34486d7b82ee2181c7f746e7426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T20:56:19.221 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:19 smithi186 podman[62126]: 2024-04-25 20:56:19.179786465 +0000 UTC m=+0.332370910 container init dd9898b1495e4f8d9de885bcffdca69bc8c5c34486d7b82ee2181c7f746e7426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T20:56:19.221 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:19 smithi186 podman[62126]: 2024-04-25 20:56:19.182204636 +0000 UTC m=+0.334789082 container start dd9898b1495e4f8d9de885bcffdca69bc8c5c34486d7b82ee2181c7f746e7426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T20:56:19.221 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mds[62145]: -- 172.21.15.186:0/433225025 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557caa399a40 con 0x557caa36d000 2024-04-25T20:56:19.221 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f[62141]: starting mds.f at 2024-04-25T20:56:19.697 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:19 smithi186 bash[62126]: dd9898b1495e4f8d9de885bcffdca69bc8c5c34486d7b82ee2181c7f746e7426 2024-04-25T20:56:19.697 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 20:56:19 smithi186 systemd[1]: Started Ceph mds.f for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:19.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mon[32529]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:19.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1477919577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:19.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:19.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:19.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:19.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:19.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:19.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:19 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:20.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:19 smithi092 ceph-mon[32524]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:20.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:19 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1477919577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:20.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:20.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:20.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:20.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:20.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:20.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:19 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:20.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:19 smithi047 ceph-mon[29195]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:20.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:19 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1477919577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:20.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:20.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:20.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:20.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:20.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:20.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:19 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:20.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:20 smithi186 ceph-mon[32529]: Deploying daemon mds.c on smithi186 2024-04-25T20:56:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:20 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.186:6834/3860377657,v1:172.21.15.186:6835/3860377657] up:boot 2024-04-25T20:56:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:20 smithi186 ceph-mon[32529]: fsmap 6 up:standby 2024-04-25T20:56:20.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:20 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:56:21.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:20 smithi092 ceph-mon[32524]: Deploying daemon mds.c on smithi186 2024-04-25T20:56:21.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:20 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.186:6834/3860377657,v1:172.21.15.186:6835/3860377657] up:boot 2024-04-25T20:56:21.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:20 smithi092 ceph-mon[32524]: fsmap 6 up:standby 2024-04-25T20:56:21.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:20 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:56:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:20 smithi047 ceph-mon[29195]: Deploying daemon mds.c on smithi186 2024-04-25T20:56:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:20 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.186:6834/3860377657,v1:172.21.15.186:6835/3860377657] up:boot 2024-04-25T20:56:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:20 smithi047 ceph-mon[29195]: fsmap 6 up:standby 2024-04-25T20:56:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:20 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:56:21.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:21.709 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:21 smithi186 systemd[1]: Starting Ceph mds.c for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:56:21.819 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:21.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:21 smithi047 ceph-mon[29195]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:21.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:21 smithi186 ceph-mon[32529]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:21.971 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:21 smithi186 podman[62435]: 2024-04-25 20:56:21.656501749 +0000 UTC m=+0.018583195 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:56:21.971 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:21 smithi186 podman[62435]: 2024-04-25 20:56:21.779857792 +0000 UTC m=+0.141939231 container create 3e8dbccef69829b0ad93dc60769edab1e70ca9569c627969b9b92fcbb4792fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T20:56:22.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:21 smithi092 ceph-mon[32524]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:22.470 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:22 smithi186 podman[62435]: 2024-04-25 20:56:22.007549296 +0000 UTC m=+0.369630733 container init 3e8dbccef69829b0ad93dc60769edab1e70ca9569c627969b9b92fcbb4792fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T20:56:22.471 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:22 smithi186 podman[62435]: 2024-04-25 20:56:22.010250168 +0000 UTC m=+0.372331606 container start 3e8dbccef69829b0ad93dc60769edab1e70ca9569c627969b9b92fcbb4792fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T20:56:22.471 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:22 smithi186 ceph-mds[62454]: -- 172.21.15.186:0/2518338514 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626e2f03a40 con 0x5626e2ed6c00 2024-04-25T20:56:22.471 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:22 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c[62450]: starting mds.c at 2024-04-25T20:56:22.471 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:22 smithi186 bash[62435]: 3e8dbccef69829b0ad93dc60769edab1e70ca9569c627969b9b92fcbb4792fa8 2024-04-25T20:56:22.471 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 20:56:22 smithi186 systemd[1]: Started Ceph mds.c for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:23.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: Deploying daemon mds.h on smithi092 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.186:6836/4095064414,v1:172.21.15.186:6837/4095064414] up:boot 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: fsmap 7 up:standby 2024-04-25T20:56:23.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:23 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:56:23.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:23.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: Deploying daemon mds.h on smithi092 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.186:6836/4095064414,v1:172.21.15.186:6837/4095064414] up:boot 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: fsmap 7 up:standby 2024-04-25T20:56:23.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:23 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:56:23.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: Deploying daemon mds.h on smithi092 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.186:6836/4095064414,v1:172.21.15.186:6837/4095064414] up:boot 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: fsmap 7 up:standby 2024-04-25T20:56:23.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:23 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:56:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 -- 172.21.15.47:0/2057793217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0564108a50 msgr2=0x7f0564108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 --2- 172.21.15.47:0/2057793217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0564108a50 0x7f0564108e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f055400b0a0 tx=0x7f055400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 -- 172.21.15.47:0/2057793217 shutdown_connections 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 --2- 172.21.15.47:0/2057793217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0564108a50 0x7f0564108e30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 --2- 172.21.15.47:0/2057793217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f056406bab0 0x7f056410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 --2- 172.21.15.47:0/2057793217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f056406b110 0x7f056406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 -- 172.21.15.47:0/2057793217 >> 172.21.15.47:0/2057793217 conn(0x7f05640788f0 msgr2=0x7f0564078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 -- 172.21.15.47:0/2057793217 shutdown_connections 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.586+0000 7f056a23b700 1 -- 172.21.15.47:0/2057793217 wait complete. 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f056a23b700 1 Processor -- start 2024-04-25T20:56:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f056a23b700 1 -- start start 2024-04-25T20:56:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f056a23b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f056406b110 0x7f0564199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f056a23b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056406bab0 0x7f0564199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f056a23b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0564108a50 0x7f056419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f056a23b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0564112bf0 con 0x7f056406bab0 2024-04-25T20:56:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f056a23b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0564112a70 con 0x7f0564108a50 2024-04-25T20:56:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f056a23b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0564112d70 con 0x7f056406b110 2024-04-25T20:56:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f0562ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056406bab0 0x7f0564199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f0562ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056406bab0 0x7f0564199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57086/0 (socket says 172.21.15.47:57086) 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f0562ffd700 1 -- 172.21.15.47:0/924234837 learned_addr learned my addr 172.21.15.47:0/924234837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.587+0000 7f05637fe700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f056406b110 0x7f0564199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0563fff700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0564108a50 0x7f056419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0562ffd700 1 -- 172.21.15.47:0/924234837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f056406b110 msgr2=0x7f0564199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0562ffd700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f056406b110 0x7f0564199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0562ffd700 1 -- 172.21.15.47:0/924234837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0564108a50 msgr2=0x7f056419d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0562ffd700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0564108a50 0x7f056419d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0562ffd700 1 -- 172.21.15.47:0/924234837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056419e130 con 0x7f056406bab0 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0563fff700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0564108a50 0x7f056419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0562ffd700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056406bab0 0x7f0564199660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e9c0 tx=0x7f054c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0560ff9700 1 -- 172.21.15.47:0/924234837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c00cbd0 con 0x7f056406bab0 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056419e420 con 0x7f056406bab0 2024-04-25T20:56:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0560ff9700 1 -- 172.21.15.47:0/924234837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f054c00cd70 con 0x7f056406bab0 2024-04-25T20:56:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f0560ff9700 1 -- 172.21.15.47:0/924234837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c005610 con 0x7f056406bab0 2024-04-25T20:56:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.588+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05641a5ca0 con 0x7f056406bab0 2024-04-25T20:56:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.589+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0564104190 con 0x7f056406bab0 2024-04-25T20:56:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.590+0000 7f0560ff9700 1 -- 172.21.15.47:0/924234837 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f054c004750 con 0x7f056406bab0 2024-04-25T20:56:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.591+0000 7f0560ff9700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f05500a3c40 0x7f05500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.591+0000 7f05637fe700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f05500a3c40 0x7f05500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.591+0000 7f0560ff9700 1 -- 172.21.15.47:0/924234837 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f054c0d02b0 con 0x7f056406bab0 2024-04-25T20:56:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.591+0000 7f05637fe700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f05500a3c40 0x7f05500a6100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0554009e60 tx=0x7f055400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.596+0000 7f0560ff9700 1 -- 172.21.15.47:0/924234837 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f054c09d130 con 0x7f056406bab0 2024-04-25T20:56:23.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.849+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f056402df80 con 0x7f056406bab0 2024-04-25T20:56:23.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.850+0000 7f0560ff9700 1 -- 172.21.15.47:0/924234837 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f054c09ca30 con 0x7f056406bab0 2024-04-25T20:56:23.851 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:23.851 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T20:56:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f05500a3c40 msgr2=0x7f05500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f05500a3c40 0x7f05500a6100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0554009e60 tx=0x7f055400b700 comp rx=0 tx=0).stop 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056406bab0 msgr2=0x7f0564199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056406bab0 0x7f0564199660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e9c0 tx=0x7f054c00ecd0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 shutdown_connections 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f05500a3c40 0x7f05500a6100 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0564108a50 0x7f056419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f056406b110 0x7f0564199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 --2- 172.21.15.47:0/924234837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056406bab0 0x7f0564199660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 >> 172.21.15.47:0/924234837 conn(0x7f05640788f0 msgr2=0x7f05640ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 shutdown_connections 2024-04-25T20:56:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:23.852+0000 7f056a23b700 1 -- 172.21.15.47:0/924234837 wait complete. 2024-04-25T20:56:23.855 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 8 2024-04-25T20:56:24.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:24 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/924234837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:24.419 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:24 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/924234837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:24.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:24 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/924234837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:25.164 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 20:56:24 smithi092 systemd[1]: Started Ceph mds.h for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:25 smithi186 ceph-mon[32529]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:25 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:25 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:25 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:25 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:25 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:25 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:25.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:25 smithi092 ceph-mon[32524]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:25.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:25 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:25 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:25 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:25 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:25.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:25 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:25.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:25 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:25.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:25 smithi047 ceph-mon[29195]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:25.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:25.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:25.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:25.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:25 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:26.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:26.439 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:26 smithi092 ceph-mon[32524]: Deploying daemon mds.i on smithi186 2024-04-25T20:56:26.439 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:26 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] up:boot 2024-04-25T20:56:26.439 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:26 smithi092 ceph-mon[32524]: fsmap 8 up:standby 2024-04-25T20:56:26.439 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:26 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:56:26.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:26 smithi186 ceph-mon[32529]: Deploying daemon mds.i on smithi186 2024-04-25T20:56:26.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:26 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] up:boot 2024-04-25T20:56:26.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:26 smithi186 ceph-mon[32529]: fsmap 8 up:standby 2024-04-25T20:56:26.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:26 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:56:26.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:26 smithi047 ceph-mon[29195]: Deploying daemon mds.i on smithi186 2024-04-25T20:56:26.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:26 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] up:boot 2024-04-25T20:56:26.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:26 smithi047 ceph-mon[29195]: fsmap 8 up:standby 2024-04-25T20:56:26.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:26 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:56:26.637 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:27.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:27 smithi186 ceph-mon[32529]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:27.471 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 systemd[1]: Starting Ceph mds.i for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:56:27.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:27 smithi092 ceph-mon[32524]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:27.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:27 smithi047 ceph-mon[29195]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:27.776 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 podman[62754]: 2024-04-25 20:56:27.39072679 +0000 UTC m=+0.017907798 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:56:27.776 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 podman[62754]: 2024-04-25 20:56:27.490527341 +0000 UTC m=+0.117708381 container create 655565dc0158c134d7909087424ec496591f972b6646b3dcb2b54ccc85459636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T20:56:27.777 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 podman[62754]: 2024-04-25 20:56:27.68443186 +0000 UTC m=+0.311612863 container init 655565dc0158c134d7909087424ec496591f972b6646b3dcb2b54ccc85459636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T20:56:27.777 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 podman[62754]: 2024-04-25 20:56:27.686991657 +0000 UTC m=+0.314172663 container start 655565dc0158c134d7909087424ec496591f972b6646b3dcb2b54ccc85459636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T20:56:27.777 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 ceph-mds[62773]: -- 172.21.15.186:0/3139186685 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab287f9a40 con 0x55ab287cc800 2024-04-25T20:56:27.777 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i[62769]: starting mds.i at 2024-04-25T20:56:27.777 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 bash[62754]: 655565dc0158c134d7909087424ec496591f972b6646b3dcb2b54ccc85459636 2024-04-25T20:56:28.220 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 20:56:27 smithi186 systemd[1]: Started Ceph mds.i for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:28.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/3366002398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 msgr2=0x7f4ad0102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:28.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/3366002398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 0x7f4ad0102990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00099c0 tx=0x7f4ac0009cd0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/3366002398 shutdown_connections 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/3366002398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0102ed0 0x7f4ad010e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/3366002398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0107c30 0x7f4ad0108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/3366002398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 0x7f4ad0102990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/3366002398 >> 172.21.15.47:0/3366002398 conn(0x7f4ad0076c60 msgr2=0x7f4ad0077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/3366002398 shutdown_connections 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.403+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/3366002398 wait complete. 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad6ab8700 1 Processor -- start 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad6ab8700 1 -- start start 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad6ab8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 0x7f4ad0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad6ab8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0102ed0 0x7f4ad0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad6ab8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0107c30 0x7f4ad0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad6ab8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0111cb0 con 0x7f4ad0102510 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad6ab8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0111b30 con 0x7f4ad0102ed0 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad6ab8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0111e30 con 0x7f4ad0107c30 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad4854700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 0x7f4ad0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad4854700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 0x7f4ad0197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51446/0 (socket says 172.21.15.47:51446) 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.404+0000 7f4ad4854700 1 -- 172.21.15.47:0/1925993800 learned_addr learned my addr 172.21.15.47:0/1925993800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad5055700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0107c30 0x7f4ad0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4acffff700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0102ed0 0x7f4ad0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad4854700 1 -- 172.21.15.47:0/1925993800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0107c30 msgr2=0x7f4ad0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad4854700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0107c30 0x7f4ad0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad4854700 1 -- 172.21.15.47:0/1925993800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0102ed0 msgr2=0x7f4ad0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad4854700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0102ed0 0x7f4ad0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad4854700 1 -- 172.21.15.47:0/1925993800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad01a7730 con 0x7f4ad0102510 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad4854700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 0x7f4ad0197ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b530 tx=0x7f4ab800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4acdffb700 1 -- 172.21.15.47:0/1925993800 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8015020 con 0x7f4ad0102510 2024-04-25T20:56:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad01a7a20 con 0x7f4ad0102510 2024-04-25T20:56:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4acdffb700 1 -- 172.21.15.47:0/1925993800 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab8011ce0 con 0x7f4ad0102510 2024-04-25T20:56:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4acdffb700 1 -- 172.21.15.47:0/1925993800 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8010d20 con 0x7f4ad0102510 2024-04-25T20:56:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.405+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01a7fb0 con 0x7f4ad0102510 2024-04-25T20:56:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.406+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad0069730 con 0x7f4ad0102510 2024-04-25T20:56:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.407+0000 7f4acdffb700 1 -- 172.21.15.47:0/1925993800 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4ab8018020 con 0x7f4ad0102510 2024-04-25T20:56:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.408+0000 7f4acdffb700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4abc0a3c40 0x7f4abc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.408+0000 7f4acdffb700 1 -- 172.21.15.47:0/1925993800 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4ab80cf790 con 0x7f4ad0102510 2024-04-25T20:56:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.408+0000 7f4acffff700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4abc0a3c40 0x7f4abc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.409+0000 7f4acffff700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4abc0a3c40 0x7f4abc0a6100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0003800 tx=0x7f4ac0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.413+0000 7f4acdffb700 1 -- 172.21.15.47:0/1925993800 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ab801f020 con 0x7f4ad0102510 2024-04-25T20:56:28.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.685+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ad01019f0 con 0x7f4ad0102510 2024-04-25T20:56:28.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.685+0000 7f4acdffb700 1 -- 172.21.15.47:0/1925993800 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7f4ab809c610 con 0x7f4ad0102510 2024-04-25T20:56:28.686 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:28.686 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T20:56:28.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.687+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4abc0a3c40 msgr2=0x7f4abc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.687+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4abc0a3c40 0x7f4abc0a6100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0003800 tx=0x7f4ac0009f10 comp rx=0 tx=0).stop 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.687+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 msgr2=0x7f4ad0197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.687+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 0x7f4ad0197ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b530 tx=0x7f4ab800b8f0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.688+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 shutdown_connections 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.688+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4abc0a3c40 0x7f4abc0a6100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.688+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ad0102510 0x7f4ad0197ec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.688+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0107c30 0x7f4ad0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.688+0000 7f4ad6ab8700 1 --2- 172.21.15.47:0/1925993800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0102ed0 0x7f4ad0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.688+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 >> 172.21.15.47:0/1925993800 conn(0x7f4ad0076c60 msgr2=0x7f4ad00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.688+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 shutdown_connections 2024-04-25T20:56:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:28.688+0000 7f4ad6ab8700 1 -- 172.21.15.47:0/1925993800 wait complete. 2024-04-25T20:56:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 10 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: Deploying daemon mds.g on smithi047 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.186:6838/60292465,v1:172.21.15.186:6839/60292465] up:boot 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: fsmap 9 up:standby 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:56:29.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:28 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1925993800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:29.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: Deploying daemon mds.g on smithi047 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.186:6838/60292465,v1:172.21.15.186:6839/60292465] up:boot 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: fsmap 9 up:standby 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:56:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:28 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1925993800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: Deploying daemon mds.g on smithi047 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.186:6838/60292465,v1:172.21.15.186:6839/60292465] up:boot 2024-04-25T20:56:29.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: fsmap 9 up:standby 2024-04-25T20:56:29.284 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:56:29.284 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:28 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1925993800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:30.764 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:30 smithi047 systemd[1]: Starting Ceph mds.g for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:56:31.019 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:30 smithi047 ceph-mon[29195]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:31.019 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:30 smithi047 podman[68480]: 2024-04-25 20:56:30.78262494 +0000 UTC m=+0.019603137 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:56:31.019 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:30 smithi047 podman[68480]: 2024-04-25 20:56:30.891774231 +0000 UTC m=+0.128752448 container create 2f3d7d4e366703580f5336921555d3f92bcdfdce477d6e94d7e7eddc60c0ab69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:56:31.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:30 smithi186 ceph-mon[32529]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:31.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:30 smithi092 ceph-mon[32524]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:31.361 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:31 smithi047 podman[68480]: 2024-04-25 20:56:31.111120734 +0000 UTC m=+0.348098917 container init 2f3d7d4e366703580f5336921555d3f92bcdfdce477d6e94d7e7eddc60c0ab69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T20:56:31.361 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:31 smithi047 podman[68480]: 2024-04-25 20:56:31.1138877 +0000 UTC m=+0.350865880 container start 2f3d7d4e366703580f5336921555d3f92bcdfdce477d6e94d7e7eddc60c0ab69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T20:56:31.361 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:31 smithi047 ceph-mds[68499]: -- 172.21.15.47:0/2633980859 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55733d853a40 con 0x55733d827000 2024-04-25T20:56:31.361 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:31 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g[68495]: starting mds.g at 2024-04-25T20:56:31.362 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:31 smithi047 bash[68480]: 2f3d7d4e366703580f5336921555d3f92bcdfdce477d6e94d7e7eddc60c0ab69 2024-04-25T20:56:31.362 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 20:56:31 smithi047 systemd[1]: Started Ceph mds.g for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:31.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:32.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:32.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:32.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:32.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: Deploying daemon mds.a on smithi047 2024-04-25T20:56:32.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.47:6838/2858318196,v1:172.21.15.47:6839/2858318196] up:boot 2024-04-25T20:56:32.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: fsmap 10 up:standby 2024-04-25T20:56:32.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:32 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:56:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:32.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:32.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:32.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:32.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: Deploying daemon mds.a on smithi047 2024-04-25T20:56:32.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.47:6838/2858318196,v1:172.21.15.47:6839/2858318196] up:boot 2024-04-25T20:56:32.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: fsmap 10 up:standby 2024-04-25T20:56:32.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:32 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:56:32.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: Deploying daemon mds.a on smithi047 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.47:6838/2858318196,v1:172.21.15.47:6839/2858318196] up:boot 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: fsmap 10 up:standby 2024-04-25T20:56:32.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:32 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:56:33.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:33 smithi047 ceph-mon[29195]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:33.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:33 smithi186 ceph-mon[32529]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:33.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:33 smithi092 ceph-mon[32524]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:33.861 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:33 smithi047 systemd[1]: Starting Ceph mds.a for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T20:56:34.236 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:34 smithi047 podman[68877]: 2024-04-25 20:56:34.143743603 +0000 UTC m=+0.442373580 container create 6b2424b7f2039a8ea6cca8dd2cdc5363acf773ea872db349d213be5cad680b36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T20:56:34.236 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:34 smithi047 podman[68877]: 2024-04-25 20:56:34.051581962 +0000 UTC m=+0.350211942 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:56:34.590 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:34 smithi047 podman[68877]: 2024-04-25 20:56:34.531696599 +0000 UTC m=+0.830326587 container init 6b2424b7f2039a8ea6cca8dd2cdc5363acf773ea872db349d213be5cad680b36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T20:56:34.590 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:34 smithi047 podman[68877]: 2024-04-25 20:56:34.534554796 +0000 UTC m=+0.833184774 container start 6b2424b7f2039a8ea6cca8dd2cdc5363acf773ea872db349d213be5cad680b36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:56:34.590 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:34 smithi047 ceph-mds[68925]: -- 172.21.15.47:0/1792242122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55797f64da40 con 0x55797f620c00 2024-04-25T20:56:34.590 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:34 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a[68916]: starting mds.a at 2024-04-25T20:56:34.590 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:34 smithi047 bash[68877]: 6b2424b7f2039a8ea6cca8dd2cdc5363acf773ea872db349d213be5cad680b36 2024-04-25T20:56:34.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 -- 172.21.15.47:0/1090861833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c06b980 msgr2=0x7f2a7c10f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/1090861833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c06b980 0x7f2a7c10f8f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b1a0 tx=0x7f2a7400b4b0 comp rx=0 tx=0).stop 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 -- 172.21.15.47:0/1090861833 shutdown_connections 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/1090861833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c06b980 0x7f2a7c10f8f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/1090861833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a7c108a20 0x7f2a7c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/1090861833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c06afe0 0x7f2a7c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 -- 172.21.15.47:0/1090861833 >> 172.21.15.47:0/1090861833 conn(0x7f2a7c0788b0 msgr2=0x7f2a7c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 -- 172.21.15.47:0/1090861833 shutdown_connections 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 -- 172.21.15.47:0/1090861833 wait complete. 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.796+0000 7f2a83e4b700 1 Processor -- start 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a83e4b700 1 -- start start 2024-04-25T20:56:34.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a83e4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c06afe0 0x7f2a7c196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a83e4b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a7c06b980 0x7f2a7c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a83e4b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c108a20 0x7f2a7c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a83e4b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c110a50 con 0x7f2a7c108a20 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a83e4b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1108d0 con 0x7f2a7c06afe0 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a83e4b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c110bd0 con 0x7f2a7c06b980 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c108a20 0x7f2a7c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c108a20 0x7f2a7c19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51798/0 (socket says 172.21.15.47:51798) 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 -- 172.21.15.47:0/4228785434 learned_addr learned my addr 172.21.15.47:0/4228785434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a813e6700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a7c06b980 0x7f2a7c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a81be7700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c06afe0 0x7f2a7c196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 -- 172.21.15.47:0/4228785434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a7c06b980 msgr2=0x7f2a7c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a7c06b980 0x7f2a7c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 -- 172.21.15.47:0/4228785434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c06afe0 msgr2=0x7f2a7c196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:34.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c06afe0 0x7f2a7c196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:34.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 -- 172.21.15.47:0/4228785434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c19bea0 con 0x7f2a7c108a20 2024-04-25T20:56:34.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.797+0000 7f2a823e8700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c108a20 0x7f2a7c19b760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400bfc0 tx=0x7f2a7400bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:34.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.798+0000 7f2a72ffd700 1 -- 172.21.15.47:0/4228785434 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74026070 con 0x7f2a7c108a20 2024-04-25T20:56:34.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.798+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c19c130 con 0x7f2a7c108a20 2024-04-25T20:56:34.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.798+0000 7f2a72ffd700 1 -- 172.21.15.47:0/4228785434 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a74007ce0 con 0x7f2a7c108a20 2024-04-25T20:56:34.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.798+0000 7f2a72ffd700 1 -- 172.21.15.47:0/4228785434 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a740104d0 con 0x7f2a7c108a20 2024-04-25T20:56:34.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.798+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c1a3a10 con 0x7f2a7c108a20 2024-04-25T20:56:34.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.799+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c104160 con 0x7f2a7c108a20 2024-04-25T20:56:34.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.800+0000 7f2a72ffd700 1 -- 172.21.15.47:0/4228785434 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a74029070 con 0x7f2a7c108a20 2024-04-25T20:56:34.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.800+0000 7f2a72ffd700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a680a3c40 0x7f2a680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:34.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.801+0000 7f2a81be7700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a680a3c40 0x7f2a680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:34.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.801+0000 7f2a72ffd700 1 -- 172.21.15.47:0/4228785434 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2a740d48f0 con 0x7f2a7c108a20 2024-04-25T20:56:34.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.801+0000 7f2a81be7700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a680a3c40 0x7f2a680a6100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a78005950 tx=0x7f2a7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:34.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:34.804+0000 7f2a72ffd700 1 -- 172.21.15.47:0/4228785434 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a740a1760 con 0x7f2a7c108a20 2024-04-25T20:56:34.861 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 20:56:34 smithi047 systemd[1]: Started Ceph mds.a for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T20:56:35.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.057+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a7c1981f0 con 0x7f2a7c108a20 2024-04-25T20:56:35.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.058+0000 7f2a72ffd700 1 -- 172.21.15.47:0/4228785434 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8818 (secure 0 0 0) 0x7f2a740a1060 con 0x7f2a7c108a20 2024-04-25T20:56:35.059 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:35.059 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T20:56:35.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a680a3c40 msgr2=0x7f2a680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:35.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a680a3c40 0x7f2a680a6100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a78005950 tx=0x7f2a7800b410 comp rx=0 tx=0).stop 2024-04-25T20:56:35.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c108a20 msgr2=0x7f2a7c19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:35.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c108a20 0x7f2a7c19b760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400bfc0 tx=0x7f2a7400bff0 comp rx=0 tx=0).stop 2024-04-25T20:56:35.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 shutdown_connections 2024-04-25T20:56:35.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a680a3c40 0x7f2a680a6100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:35.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a7c108a20 0x7f2a7c19b760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:35.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c06afe0 0x7f2a7c196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:35.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 --2- 172.21.15.47:0/4228785434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a7c06b980 0x7f2a7c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:35.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 >> 172.21.15.47:0/4228785434 conn(0x7f2a7c0788b0 msgr2=0x7f2a7c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:35.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 shutdown_connections 2024-04-25T20:56:35.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:35.060+0000 7f2a83e4b700 1 -- 172.21.15.47:0/4228785434 wait complete. 2024-04-25T20:56:35.062 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2024-04-25T20:56:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: Deploying daemon mds.b on smithi092 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/4228785434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.47:6840/622157648,v1:172.21.15.47:6841/622157648] up:boot 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: fsmap 11 up:standby 2024-04-25T20:56:35.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:35 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:56:36.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: Deploying daemon mds.b on smithi092 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/4228785434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.47:6840/622157648,v1:172.21.15.47:6841/622157648] up:boot 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: fsmap 11 up:standby 2024-04-25T20:56:36.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:35 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: Deploying daemon mds.b on smithi092 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/4228785434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.47:6840/622157648,v1:172.21.15.47:6841/622157648] up:boot 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: fsmap 11 up:standby 2024-04-25T20:56:36.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:35 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:56:37.762 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:37.914 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:37 smithi047 ceph-mon[29195]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:37.915 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:37 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.915 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:37 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.915 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:37 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.915 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:37 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.915 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:37 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:37 smithi186 ceph-mon[32529]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:37 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:37 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:37 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:37 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:37.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:37 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:38.028 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:37 smithi092 ceph-mon[32524]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:37 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:37 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:37 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:37 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:37 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:38.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:38 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] up:boot 2024-04-25T20:56:38.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:38 smithi047 ceph-mon[29195]: fsmap 12 up:standby 2024-04-25T20:56:38.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:38 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:56:38.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:38 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:38.778 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:38 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] up:boot 2024-04-25T20:56:38.778 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:38 smithi092 ceph-mon[32524]: fsmap 12 up:standby 2024-04-25T20:56:38.778 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:56:38.778 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:38 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:38.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:38 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] up:boot 2024-04-25T20:56:38.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:38 smithi186 ceph-mon[32529]: fsmap 12 up:standby 2024-04-25T20:56:38.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:38 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:56:38.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:38 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 -- 172.21.15.47:0/4276713067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe620108a20 msgr2=0x7fe620108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 --2- 172.21.15.47:0/4276713067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe620108a20 0x7fe620108e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe614009990 tx=0x7fe614009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 -- 172.21.15.47:0/4276713067 shutdown_connections 2024-04-25T20:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 --2- 172.21.15.47:0/4276713067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe620108a20 0x7fe620108e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 --2- 172.21.15.47:0/4276713067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe62006b980 0x7fe62010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 --2- 172.21.15.47:0/4276713067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe62006afe0 0x7fe62006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 -- 172.21.15.47:0/4276713067 >> 172.21.15.47:0/4276713067 conn(0x7fe6200788b0 msgr2=0x7fe620078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 -- 172.21.15.47:0/4276713067 shutdown_connections 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.697+0000 7fe626f8a700 1 -- 172.21.15.47:0/4276713067 wait complete. 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe626f8a700 1 Processor -- start 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe626f8a700 1 -- start start 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe626f8a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe62006afe0 0x7fe6201990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe626f8a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe62006b980 0x7fe620199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe626f8a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe620108a20 0x7fe62019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe626f8a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe620112bc0 con 0x7fe620108a20 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe626f8a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe620112a40 con 0x7fe62006b980 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe626f8a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe620112d40 con 0x7fe62006afe0 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe624d26700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe62006afe0 0x7fe6201990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe624d26700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe62006afe0 0x7fe6201990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:38376/0 (socket says 172.21.15.47:38376) 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe624d26700 1 -- 172.21.15.47:0/3419145938 learned_addr learned my addr 172.21.15.47:0/3419145938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe61ffff700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe62006b980 0x7fe620199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.698+0000 7fe625527700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe620108a20 0x7fe62019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe624d26700 1 -- 172.21.15.47:0/3419145938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe62006b980 msgr2=0x7fe620199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe624d26700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe62006b980 0x7fe620199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe624d26700 1 -- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe620108a20 msgr2=0x7fe62019d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe624d26700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe620108a20 0x7fe62019d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe624d26700 1 -- 172.21.15.47:0/3419145938 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62019e100 con 0x7fe62006afe0 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe624d26700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe62006afe0 0x7fe6201990f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe614000c00 tx=0x7fe61400bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe61dffb700 1 -- 172.21.15.47:0/3419145938 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614027070 con 0x7fe62006afe0 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe61dffb700 1 -- 172.21.15.47:0/3419145938 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe614022470 con 0x7fe62006afe0 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62019e390 con 0x7fe62006afe0 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe61dffb700 1 -- 172.21.15.47:0/3419145938 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614017650 con 0x7fe62006afe0 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.699+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6201a5c70 con 0x7fe62006afe0 2024-04-25T20:56:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.700+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe620069730 con 0x7fe62006afe0 2024-04-25T20:56:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.701+0000 7fe61dffb700 1 -- 172.21.15.47:0/3419145938 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe61402a070 con 0x7fe62006afe0 2024-04-25T20:56:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.702+0000 7fe61dffb700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6100a3cc0 0x7fe6100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.702+0000 7fe61ffff700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6100a3cc0 0x7fe6100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.702+0000 7fe61dffb700 1 -- 172.21.15.47:0/3419145938 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe6140d0bb0 con 0x7fe62006afe0 2024-04-25T20:56:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.703+0000 7fe61ffff700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6100a3cc0 0x7fe6100a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe620078f90 tx=0x7fe60c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.707+0000 7fe61dffb700 1 -- 172.21.15.47:0/3419145938 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe614013070 con 0x7fe62006afe0 2024-04-25T20:56:39.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:39 smithi047 ceph-mon[29195]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:39.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:39.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:39.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:39.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:39 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:39.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:39 smithi186 ceph-mon[32529]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:39.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:39.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:39.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:39.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:39 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:39.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.992+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe62002df80 con 0x7fe62006afe0 2024-04-25T20:56:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.993+0000 7fe61dffb700 1 -- 172.21.15.47:0/3419145938 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7fe61409da20 con 0x7fe62006afe0 2024-04-25T20:56:39.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:39.994 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T20:56:39.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6100a3cc0 msgr2=0x7fe6100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:39.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6100a3cc0 0x7fe6100a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe620078f90 tx=0x7fe60c009260 comp rx=0 tx=0).stop 2024-04-25T20:56:39.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe62006afe0 msgr2=0x7fe6201990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:39.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe62006afe0 0x7fe6201990f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe614000c00 tx=0x7fe61400bfd0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 shutdown_connections 2024-04-25T20:56:39.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6100a3cc0 0x7fe6100a6180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe620108a20 0x7fe62019d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe62006afe0 0x7fe6201990f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 --2- 172.21.15.47:0/3419145938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe62006b980 0x7fe620199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 >> 172.21.15.47:0/3419145938 conn(0x7fe6200788b0 msgr2=0x7fe6200ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:39.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 shutdown_connections 2024-04-25T20:56:39.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:39.996+0000 7fe626f8a700 1 -- 172.21.15.47:0/3419145938 wait complete. 2024-04-25T20:56:39.998 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 13 2024-04-25T20:56:40.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:39 smithi092 ceph-mon[32524]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:40.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:40.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:40.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:40.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:39 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:40.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:40 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3419145938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:40.896 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-25T20:56:40.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:40 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3419145938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:41.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:40 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3419145938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:41.169 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:41.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:41 smithi047 ceph-mon[29195]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:41.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:41 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:41.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:41 smithi186 ceph-mon[32529]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:41.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:41 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:42.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:41 smithi092 ceph-mon[32524]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:42.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:41 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:42.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 -- 172.21.15.47:0/871142389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2401019d0 msgr2=0x7fb24010f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:42.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 --2- 172.21.15.47:0/871142389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2401019d0 0x7fb24010f910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb23800b0a0 tx=0x7fb23800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:42.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 -- 172.21.15.47:0/871142389 shutdown_connections 2024-04-25T20:56:42.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 --2- 172.21.15.47:0/871142389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2401019d0 0x7fb24010f910 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 --2- 172.21.15.47:0/871142389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb240108ad0 0x7fb240108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 --2- 172.21.15.47:0/871142389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb240101030 0x7fb240101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 -- 172.21.15.47:0/871142389 >> 172.21.15.47:0/871142389 conn(0x7fb240077fb0 msgr2=0x7fb2400ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 -- 172.21.15.47:0/871142389 shutdown_connections 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.814+0000 7fb247456700 1 -- 172.21.15.47:0/871142389 wait complete. 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb247456700 1 Processor -- start 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb247456700 1 -- start start 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb247456700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb240101030 0x7fb24019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb247456700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2401019d0 0x7fb24019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb247456700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240108ad0 0x7fb24019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb247456700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb240112be0 con 0x7fb240108ad0 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb247456700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb240112a60 con 0x7fb2401019d0 2024-04-25T20:56:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb247456700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb240112d60 con 0x7fb240101030 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb2459f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240108ad0 0x7fb24019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb2459f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240108ad0 0x7fb24019fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51824/0 (socket says 172.21.15.47:51824) 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb2459f3700 1 -- 172.21.15.47:0/2590804648 learned_addr learned my addr 172.21.15.47:0/2590804648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb2451f2700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb240101030 0x7fb24019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.815+0000 7fb2449f1700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2401019d0 0x7fb24019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2459f3700 1 -- 172.21.15.47:0/2590804648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb240101030 msgr2=0x7fb24019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2459f3700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb240101030 0x7fb24019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2459f3700 1 -- 172.21.15.47:0/2590804648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2401019d0 msgr2=0x7fb24019b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2459f3700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2401019d0 0x7fb24019b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2459f3700 1 -- 172.21.15.47:0/2590804648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2401a0280 con 0x7fb240108ad0 2024-04-25T20:56:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2459f3700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240108ad0 0x7fb24019fb40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2380062a0 tx=0x7fb238004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2367fc700 1 -- 172.21.15.47:0/2590804648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238021070 con 0x7fb240108ad0 2024-04-25T20:56:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2401a0510 con 0x7fb240108ad0 2024-04-25T20:56:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2367fc700 1 -- 172.21.15.47:0/2590804648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb238007e30 con 0x7fb240108ad0 2024-04-25T20:56:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb2367fc700 1 -- 172.21.15.47:0/2590804648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2380046d0 con 0x7fb240108ad0 2024-04-25T20:56:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.816+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2401a7df0 con 0x7fb240108ad0 2024-04-25T20:56:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.817+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb240104350 con 0x7fb240108ad0 2024-04-25T20:56:42.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.818+0000 7fb2367fc700 1 -- 172.21.15.47:0/2590804648 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb238007980 con 0x7fb240108ad0 2024-04-25T20:56:42.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.819+0000 7fb2367fc700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb22c0a3c40 0x7fb22c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:42.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.819+0000 7fb2451f2700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb22c0a3c40 0x7fb22c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:42.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.819+0000 7fb2367fc700 1 -- 172.21.15.47:0/2590804648 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb23800f020 con 0x7fb240108ad0 2024-04-25T20:56:42.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.820+0000 7fb2451f2700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb22c0a3c40 0x7fb22c0a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb23c007b60 tx=0x7fb23c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:42.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:42.827+0000 7fb2367fc700 1 -- 172.21.15.47:0/2590804648 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb23802a020 con 0x7fb240108ad0 2024-04-25T20:56:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.040+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb2401a7fc0 con 0x7fb240108ad0 2024-04-25T20:56:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.041+0000 7fb2367fc700 1 -- 172.21.15.47:0/2590804648 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fb23809e670 con 0x7fb240108ad0 2024-04-25T20:56:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:43.042 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-25T20:56:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T20:56:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-25T20:56:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-25T20:56:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T20:56:43.042 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-25T20:56:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb22c0a3c40 msgr2=0x7fb22c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb22c0a3c40 0x7fb22c0a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb23c007b60 tx=0x7fb23c0073d0 comp rx=0 tx=0).stop 2024-04-25T20:56:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240108ad0 msgr2=0x7fb24019fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240108ad0 0x7fb24019fb40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2380062a0 tx=0x7fb238004c80 comp rx=0 tx=0).stop 2024-04-25T20:56:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 shutdown_connections 2024-04-25T20:56:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb22c0a3c40 0x7fb22c0a6100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240108ad0 0x7fb24019fb40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:43.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb240101030 0x7fb24019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:43.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 --2- 172.21.15.47:0/2590804648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2401019d0 0x7fb24019b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:43.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 >> 172.21.15.47:0/2590804648 conn(0x7fb240077fb0 msgr2=0x7fb2400ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:43.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 shutdown_connections 2024-04-25T20:56:43.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:43.044+0000 7fb247456700 1 -- 172.21.15.47:0/2590804648 wait complete. 2024-04-25T20:56:43.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:43 smithi047 ceph-mon[29195]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:43.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:43.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:43.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:43 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:43.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:43 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2590804648' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T20:56:43.710 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T20:56:43.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T20:56:43.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:43 smithi186 ceph-mon[32529]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:43.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:43.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:43.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:43 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:43.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:43 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2590804648' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T20:56:43.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:43 smithi092 ceph-mon[32524]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:43.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:43.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:43.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:43.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:43 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:43.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:43 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2590804648' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T20:56:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:45.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:45 smithi092 ceph-mon[32524]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:45.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:45 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:45 smithi047 ceph-mon[29195]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:45.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:45 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:45 smithi186 ceph-mon[32529]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:45.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:45.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:45 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:46.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.093+0000 7f17d876e700 1 -- 172.21.15.47:0/262478737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006a750 msgr2=0x7f17d006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:46.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.093+0000 7f17d876e700 1 --2- 172.21.15.47:0/262478737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006a750 0x7f17d006abb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009990 tx=0x7f17c0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:46.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 -- 172.21.15.47:0/262478737 shutdown_connections 2024-04-25T20:56:46.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 --2- 172.21.15.47:0/262478737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17d006b0f0 0x7f17d010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:46.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 --2- 172.21.15.47:0/262478737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d0108a30 0x7f17d0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:46.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 --2- 172.21.15.47:0/262478737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006a750 0x7f17d006abb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:46.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 -- 172.21.15.47:0/262478737 >> 172.21.15.47:0/262478737 conn(0x7f17d00788f0 msgr2=0x7f17d0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:46.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 -- 172.21.15.47:0/262478737 shutdown_connections 2024-04-25T20:56:46.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 -- 172.21.15.47:0/262478737 wait complete. 2024-04-25T20:56:46.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 Processor -- start 2024-04-25T20:56:46.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.094+0000 7f17d876e700 1 -- start start 2024-04-25T20:56:46.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d876e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17d006a750 0x7f17d01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:46.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d876e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006b0f0 0x7f17d01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d876e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d0108a30 0x7f17d019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d876e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0112c70 con 0x7f17d006b0f0 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d876e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0112af0 con 0x7f17d0108a30 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d876e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0112df0 con 0x7f17d006a750 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006b0f0 0x7f17d01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006b0f0 0x7f17d01996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44988/0 (socket says 172.21.15.47:44988) 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 -- 172.21.15.47:0/124893548 learned_addr learned my addr 172.21.15.47:0/124893548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d6d0b700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d0108a30 0x7f17d019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d650a700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17d006a750 0x7f17d01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 -- 172.21.15.47:0/124893548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17d006a750 msgr2=0x7f17d01991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17d006a750 0x7f17d01991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 -- 172.21.15.47:0/124893548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d0108a30 msgr2=0x7f17d019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:46.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d0108a30 0x7f17d019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:46.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 -- 172.21.15.47:0/124893548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d019e160 con 0x7f17d006b0f0 2024-04-25T20:56:46.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d5d09700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006b0f0 0x7f17d01996e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f17c00057f0 tx=0x7f17c0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:46.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17c77fe700 1 -- 172.21.15.47:0/124893548 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0027070 con 0x7f17d006b0f0 2024-04-25T20:56:46.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d019e3f0 con 0x7f17d006b0f0 2024-04-25T20:56:46.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17c77fe700 1 -- 172.21.15.47:0/124893548 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c0004020 con 0x7f17d006b0f0 2024-04-25T20:56:46.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.095+0000 7f17c77fe700 1 -- 172.21.15.47:0/124893548 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0022cf0 con 0x7f17d006b0f0 2024-04-25T20:56:46.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.096+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d01a5cd0 con 0x7f17d006b0f0 2024-04-25T20:56:46.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.096+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d0069730 con 0x7f17d006b0f0 2024-04-25T20:56:46.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.098+0000 7f17c77fe700 1 -- 172.21.15.47:0/124893548 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f17c002a070 con 0x7f17d006b0f0 2024-04-25T20:56:46.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.098+0000 7f17c77fe700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f17bc0a3c40 0x7f17bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:46.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.098+0000 7f17d650a700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f17bc0a3c40 0x7f17bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:46.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.098+0000 7f17c77fe700 1 -- 172.21.15.47:0/124893548 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f17c00d15c0 con 0x7f17d006b0f0 2024-04-25T20:56:46.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.099+0000 7f17d650a700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f17bc0a3c40 0x7f17bc0a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005950 tx=0x7f17cc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:46.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.102+0000 7f17c77fe700 1 -- 172.21.15.47:0/124893548 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17c0031020 con 0x7f17d006b0f0 2024-04-25T20:56:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:46.323+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f17d019a4e0 con 0x7f17d006b0f0 2024-04-25T20:56:47.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.235+0000 7f17c77fe700 1 -- 172.21.15.47:0/124893548 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f17d019a4e0 con 0x7f17d006b0f0 2024-04-25T20:56:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f17bc0a3c40 msgr2=0x7f17bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f17bc0a3c40 0x7f17bc0a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005950 tx=0x7f17cc00a300 comp rx=0 tx=0).stop 2024-04-25T20:56:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006b0f0 msgr2=0x7f17d01996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006b0f0 0x7f17d01996e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f17c00057f0 tx=0x7f17c0003680 comp rx=0 tx=0).stop 2024-04-25T20:56:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 shutdown_connections 2024-04-25T20:56:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f17bc0a3c40 0x7f17bc0a6100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d0108a30 0x7f17d019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17d006a750 0x7f17d01991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 --2- 172.21.15.47:0/124893548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d006b0f0 0x7f17d01996e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.238+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 >> 172.21.15.47:0/124893548 conn(0x7f17d00788f0 msgr2=0x7f17d00ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:47.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.239+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 shutdown_connections 2024-04-25T20:56:47.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:47.239+0000 7f17d876e700 1 -- 172.21.15.47:0/124893548 wait complete. 2024-04-25T20:56:47.242 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-25T20:56:47.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:47 smithi092 ceph-mon[32524]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:47 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/124893548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T20:56:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:47.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:47 smithi047 ceph-mon[29195]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:47.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:47 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/124893548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T20:56:47.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:47.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:47.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:47 smithi186 ceph-mon[32529]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:47.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:47 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/124893548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T20:56:47.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:47.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:47.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:56:47.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T20:56:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:48.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:48 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/124893548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T20:56:48.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:48 smithi092 ceph-mon[32524]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T20:56:48.599 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:48 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/124893548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T20:56:48.599 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:48 smithi047 ceph-mon[29195]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T20:56:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:48 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/124893548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T20:56:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:48 smithi186 ceph-mon[32529]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T20:56:49.502 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:49 smithi047 ceph-mon[29195]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:49.502 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:49 smithi047 ceph-mon[29195]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T20:56:49.502 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:49 smithi047 ceph-mon[29195]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T20:56:49.519 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:49 smithi092 ceph-mon[32524]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:49.519 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:49 smithi092 ceph-mon[32524]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T20:56:49.519 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:49 smithi092 ceph-mon[32524]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T20:56:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 -- 172.21.15.47:0/4116034598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7978108a70 msgr2=0x7f7978108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 --2- 172.21.15.47:0/4116034598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7978108a70 0x7f7978108e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7974009990 tx=0x7f7974009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 -- 172.21.15.47:0/4116034598 shutdown_connections 2024-04-25T20:56:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 --2- 172.21.15.47:0/4116034598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7978108a70 0x7f7978108e50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 --2- 172.21.15.47:0/4116034598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f797806bb30 0x7f797810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 --2- 172.21.15.47:0/4116034598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797806b170 0x7f797806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 -- 172.21.15.47:0/4116034598 >> 172.21.15.47:0/4116034598 conn(0x7f79780777b0 msgr2=0x7f7978077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 -- 172.21.15.47:0/4116034598 shutdown_connections 2024-04-25T20:56:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.687+0000 7f797f2de700 1 -- 172.21.15.47:0/4116034598 wait complete. 2024-04-25T20:56:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797f2de700 1 Processor -- start 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797f2de700 1 -- start start 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797f2de700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f797806b170 0x7f7978101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797f2de700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797806bb30 0x7f79781022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797f2de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978108a70 0x7f79781027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797f2de700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7978112c30 con 0x7f797806b170 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797f2de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7978112ab0 con 0x7f7978108a70 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797f2de700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7978112db0 con 0x7f797806bb30 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d07a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f797806b170 0x7f7978101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d07a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f797806b170 0x7f7978101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45004/0 (socket says 172.21.15.47:45004) 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d07a700 1 -- 172.21.15.47:0/640931123 learned_addr learned my addr 172.21.15.47:0/640931123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d87b700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978108a70 0x7f79781027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797c879700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797806bb30 0x7f79781022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d07a700 1 -- 172.21.15.47:0/640931123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797806bb30 msgr2=0x7f79781022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d07a700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797806bb30 0x7f79781022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d07a700 1 -- 172.21.15.47:0/640931123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978108a70 msgr2=0x7f79781027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d07a700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978108a70 0x7f79781027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.688+0000 7f797d07a700 1 -- 172.21.15.47:0/640931123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79781a7630 con 0x7f797806b170 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.689+0000 7f797d07a700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f797806b170 0x7f7978101d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f797400bf20 tx=0x7f797400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.689+0000 7f796e7fc700 1 -- 172.21.15.47:0/640931123 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974027070 con 0x7f797806b170 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.689+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79781a78c0 con 0x7f797806b170 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.689+0000 7f796e7fc700 1 -- 172.21.15.47:0/640931123 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797400fdd0 con 0x7f797806b170 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.689+0000 7f796e7fc700 1 -- 172.21.15.47:0/640931123 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79740173f0 con 0x7f797806b170 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.689+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79781a7d70 con 0x7f797806b170 2024-04-25T20:56:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.690+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79781041a0 con 0x7f797806b170 2024-04-25T20:56:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.691+0000 7f796e7fc700 1 -- 172.21.15.47:0/640931123 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f797402a070 con 0x7f797806b170 2024-04-25T20:56:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.692+0000 7f796e7fc700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f79640a3c40 0x7f79640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.692+0000 7f797c879700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f79640a3c40 0x7f79640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.692+0000 7f796e7fc700 1 -- 172.21.15.47:0/640931123 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f79740d1190 con 0x7f797806b170 2024-04-25T20:56:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.692+0000 7f797c879700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f79640a3c40 0x7f79640a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7978077e60 tx=0x7f796800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.696+0000 7f796e7fc700 1 -- 172.21.15.47:0/640931123 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f797405eff0 con 0x7f797806b170 2024-04-25T20:56:49.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:49 smithi186 ceph-mon[32529]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:49.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:49 smithi186 ceph-mon[32529]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T20:56:49.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:49 smithi186 ceph-mon[32529]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T20:56:49.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:49.906+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f797802df80 con 0x7f797806b170 2024-04-25T20:56:50.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.251+0000 7f796e7fc700 1 -- 172.21.15.47:0/640931123 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f7974013070 con 0x7f797806b170 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f79640a3c40 msgr2=0x7f79640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f79640a3c40 0x7f79640a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7978077e60 tx=0x7f796800a300 comp rx=0 tx=0).stop 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f797806b170 msgr2=0x7f7978101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f797806b170 0x7f7978101d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f797400bf20 tx=0x7f797400f460 comp rx=0 tx=0).stop 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 shutdown_connections 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f79640a3c40 0x7f79640a6100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f797806b170 0x7f7978101d60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978108a70 0x7f79781027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 --2- 172.21.15.47:0/640931123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797806bb30 0x7f79781022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 >> 172.21.15.47:0/640931123 conn(0x7f79780777b0 msgr2=0x7f79781009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:50.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 shutdown_connections 2024-04-25T20:56:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:50.254+0000 7f797f2de700 1 -- 172.21.15.47:0/640931123 wait complete. 2024-04-25T20:56:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-25T20:56:50.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:50 smithi092 ceph-mon[32524]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T20:56:50.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:50 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/640931123' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T20:56:50.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:50 smithi047 ceph-mon[29195]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T20:56:50.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:50 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/640931123' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T20:56:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:50 smithi186 ceph-mon[32529]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T20:56:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:50 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/640931123' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T20:56:50.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T20:56:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:51.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:51 smithi092 ceph-mon[32524]: pgmap v287: 33 pgs: 1 creating+activating, 8 creating+peering, 14 active+clean, 10 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:51.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:51 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/640931123' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T20:56:51.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:51 smithi092 ceph-mon[32524]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T20:56:51.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:51 smithi047 ceph-mon[29195]: pgmap v287: 33 pgs: 1 creating+activating, 8 creating+peering, 14 active+clean, 10 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:51.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:51 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/640931123' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T20:56:51.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:51 smithi047 ceph-mon[29195]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T20:56:51.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:51 smithi186 ceph-mon[32529]: pgmap v287: 33 pgs: 1 creating+activating, 8 creating+peering, 14 active+clean, 10 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:51.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:51 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/640931123' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T20:56:51.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:51 smithi186 ceph-mon[32529]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T20:56:52.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:52 smithi092 ceph-mon[32524]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T20:56:52.606 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:52 smithi047 ceph-mon[29195]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T20:56:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:52 smithi186 ceph-mon[32529]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T20:56:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 -- 172.21.15.47:0/3886280699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 msgr2=0x7fbcc410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 --2- 172.21.15.47:0/3886280699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 0x7fbcc410f8e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00b0a0 tx=0x7fbcbc00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 -- 172.21.15.47:0/3886280699 shutdown_connections 2024-04-25T20:56:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 --2- 172.21.15.47:0/3886280699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 0x7fbcc410f8e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 --2- 172.21.15.47:0/3886280699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4108a80 0x7fbcc4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 --2- 172.21.15.47:0/3886280699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcc4102a90 0x7fbcc4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 -- 172.21.15.47:0/3886280699 >> 172.21.15.47:0/3886280699 conn(0x7fbcc4079bf0 msgr2=0x7fbcc407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 -- 172.21.15.47:0/3886280699 shutdown_connections 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.812+0000 7fbccaf10700 1 -- 172.21.15.47:0/3886280699 wait complete. 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbccaf10700 1 Processor -- start 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbccaf10700 1 -- start start 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbccaf10700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4102a90 0x7fbcc419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbccaf10700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 0x7fbcc419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbccaf10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcc4108a80 0x7fbcc419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbccaf10700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4112c30 con 0x7fbcc4103450 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbccaf10700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4112ab0 con 0x7fbcc4108a80 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbccaf10700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4112db0 con 0x7fbcc4102a90 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 0x7fbcc419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 0x7fbcc419b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45016/0 (socket says 172.21.15.47:45016) 2024-04-25T20:56:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbcc3fff700 1 -- 172.21.15.47:0/374954971 learned_addr learned my addr 172.21.15.47:0/374954971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.813+0000 7fbcc94ad700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcc4108a80 0x7fbcc419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc8cac700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4102a90 0x7fbcc419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc3fff700 1 -- 172.21.15.47:0/374954971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4102a90 msgr2=0x7fbcc419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc3fff700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4102a90 0x7fbcc419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc3fff700 1 -- 172.21.15.47:0/374954971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcc4108a80 msgr2=0x7fbcc419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc3fff700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcc4108a80 0x7fbcc419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc3fff700 1 -- 172.21.15.47:0/374954971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc41a0310 con 0x7fbcc4103450 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc3fff700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 0x7fbcc419b840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d950 tx=0x7fbcb000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc1ffb700 1 -- 172.21.15.47:0/374954971 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0016070 con 0x7fbcc4103450 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc1ffb700 1 -- 172.21.15.47:0/374954971 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb000fd40 con 0x7fbcc4103450 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc41a0600 con 0x7fbcc4103450 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbcc1ffb700 1 -- 172.21.15.47:0/374954971 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000e3b0 con 0x7fbcc4103450 2024-04-25T20:56:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.814+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc41a7e80 con 0x7fbcc4103450 2024-04-25T20:56:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.815+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc4069730 con 0x7fbcc4103450 2024-04-25T20:56:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.817+0000 7fbcc1ffb700 1 -- 172.21.15.47:0/374954971 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbcb0019070 con 0x7fbcc4103450 2024-04-25T20:56:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.817+0000 7fbcc1ffb700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbcb40a3c40 0x7fbcb40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.817+0000 7fbcc8cac700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbcb40a3c40 0x7fbcb40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.817+0000 7fbcc1ffb700 1 -- 172.21.15.47:0/374954971 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7fbcb00cfd00 con 0x7fbcc4103450 2024-04-25T20:56:52.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.818+0000 7fbcc8cac700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbcb40a3c40 0x7fbcb40a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80045b0 tx=0x7fbcb800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:52.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:52.822+0000 7fbcc1ffb700 1 -- 172.21.15.47:0/374954971 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcb009c890 con 0x7fbcc4103450 2024-04-25T20:56:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.084+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbcc41a8050 con 0x7fbcc4103450 2024-04-25T20:56:53.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.275+0000 7fbcc1ffb700 1 -- 172.21.15.47:0/374954971 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fbcb009c190 con 0x7fbcc4103450 2024-04-25T20:56:53.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbcb40a3c40 msgr2=0x7fbcb40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:53.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbcb40a3c40 0x7fbcb40a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80045b0 tx=0x7fbcb800b3c0 comp rx=0 tx=0).stop 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 msgr2=0x7fbcc419b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 0x7fbcc419b840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d950 tx=0x7fbcb000dc60 comp rx=0 tx=0).stop 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 shutdown_connections 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbcb40a3c40 0x7fbcb40a6100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcc4108a80 0x7fbcc419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4102a90 0x7fbcc419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 --2- 172.21.15.47:0/374954971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc4103450 0x7fbcc419b840 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 >> 172.21.15.47:0/374954971 conn(0x7fbcc4079bf0 msgr2=0x7fbcc410dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.277+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 shutdown_connections 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:53.278+0000 7fbccaf10700 1 -- 172.21.15.47:0/374954971 wait complete. 2024-04-25T20:56:53.279 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T20:56:53.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:53 smithi047 ceph-mon[29195]: pgmap v290: 97 pgs: 1 creating+activating, 22 creating+peering, 24 active+clean, 50 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:53.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:53 smithi047 ceph-mon[29195]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T20:56:53.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:53 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/374954971' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T20:56:53.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:53 smithi047 ceph-mon[29195]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T20:56:53.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:53 smithi092 ceph-mon[32524]: pgmap v290: 97 pgs: 1 creating+activating, 22 creating+peering, 24 active+clean, 50 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:53.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:53 smithi092 ceph-mon[32524]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T20:56:53.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:53 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/374954971' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T20:56:53.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:53 smithi092 ceph-mon[32524]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T20:56:53.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:53 smithi186 ceph-mon[32529]: pgmap v290: 97 pgs: 1 creating+activating, 22 creating+peering, 24 active+clean, 50 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:53.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:53 smithi186 ceph-mon[32529]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T20:56:53.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:53 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/374954971' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T20:56:53.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:53 smithi186 ceph-mon[32529]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T20:56:53.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:56:54.235 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:54.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:54 smithi092 ceph-mon[32524]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T20:56:54.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:54 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/374954971' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T20:56:54.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:54 smithi092 ceph-mon[32524]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T20:56:54.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:54 smithi092 ceph-mon[32524]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T20:56:54.598 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:54 smithi047 ceph-mon[29195]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T20:56:54.598 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:54 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/374954971' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T20:56:54.598 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:54 smithi047 ceph-mon[29195]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T20:56:54.598 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:54 smithi047 ceph-mon[29195]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T20:56:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:54 smithi186 ceph-mon[32529]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T20:56:54.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:54 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/374954971' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T20:56:54.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:54 smithi186 ceph-mon[32529]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T20:56:54.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:54 smithi186 ceph-mon[32529]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T20:56:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:55 smithi047 ceph-mon[29195]: pgmap v293: 97 pgs: 14 creating+peering, 56 active+clean, 27 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:55 smithi047 ceph-mon[29195]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T20:56:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:55 smithi047 ceph-mon[29195]: Cluster is now healthy 2024-04-25T20:56:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:55 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] up:active 2024-04-25T20:56:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:55 smithi047 ceph-mon[29195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:56:55.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:55 smithi047 ceph-mon[29195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:56:55.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:55 smithi092 ceph-mon[32524]: pgmap v293: 97 pgs: 14 creating+peering, 56 active+clean, 27 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:55.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:55 smithi092 ceph-mon[32524]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T20:56:55.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:55 smithi092 ceph-mon[32524]: Cluster is now healthy 2024-04-25T20:56:55.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:55 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] up:active 2024-04-25T20:56:55.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:55 smithi092 ceph-mon[32524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:56:55.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:55 smithi092 ceph-mon[32524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:56:55.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:55 smithi186 ceph-mon[32529]: pgmap v293: 97 pgs: 14 creating+peering, 56 active+clean, 27 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:56:55.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:55 smithi186 ceph-mon[32529]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T20:56:55.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:55 smithi186 ceph-mon[32529]: Cluster is now healthy 2024-04-25T20:56:55.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:55 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] up:active 2024-04-25T20:56:55.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:55 smithi186 ceph-mon[32529]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:56:55.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:55 smithi186 ceph-mon[32529]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:56:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 -- 172.21.15.47:0/680967854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18107990 msgr2=0x7f5f18107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 --2- 172.21.15.47:0/680967854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18107990 0x7f5f18107d70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009990 tx=0x7f5f14009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 -- 172.21.15.47:0/680967854 shutdown_connections 2024-04-25T20:56:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 --2- 172.21.15.47:0/680967854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18107990 0x7f5f18107d70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 --2- 172.21.15.47:0/680967854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f18102300 0x7f5f1810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 --2- 172.21.15.47:0/680967854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f18101940 0x7f5f18101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 -- 172.21.15.47:0/680967854 >> 172.21.15.47:0/680967854 conn(0x7f5f180788f0 msgr2=0x7f5f18078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 -- 172.21.15.47:0/680967854 shutdown_connections 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.892+0000 7f5f20150700 1 -- 172.21.15.47:0/680967854 wait complete. 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f20150700 1 Processor -- start 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f20150700 1 -- start start 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f20150700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f18101940 0x7f5f18199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f20150700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18102300 0x7f5f1819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f20150700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f18107990 0x7f5f181a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f20150700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111b30 con 0x7f5f18102300 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f20150700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181119b0 con 0x7f5f18107990 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f20150700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111cb0 con 0x7f5f18101940 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f1d6eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18102300 0x7f5f1819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f1d6eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18102300 0x7f5f1819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42812/0 (socket says 172.21.15.47:42812) 2024-04-25T20:56:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f1d6eb700 1 -- 172.21.15.47:0/317088502 learned_addr learned my addr 172.21.15.47:0/317088502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:55.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f1deec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f18101940 0x7f5f18199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.893+0000 7f5f1e6ed700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f18107990 0x7f5f181a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f1e6ed700 1 -- 172.21.15.47:0/317088502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f18101940 msgr2=0x7f5f18199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f1e6ed700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f18101940 0x7f5f18199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f1e6ed700 1 -- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18102300 msgr2=0x7f5f1819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f1e6ed700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18102300 0x7f5f1819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f1e6ed700 1 -- 172.21.15.47:0/317088502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181a5570 con 0x7f5f18107990 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f1deec700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f18101940 0x7f5f18199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f1d6eb700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18102300 0x7f5f1819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f1e6ed700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f18107990 0x7f5f181a5030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000efc0 tx=0x7f5f1000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f0effd700 1 -- 172.21.15.47:0/317088502 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100092e0 con 0x7f5f18107990 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f181a5740 con 0x7f5f18107990 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f0effd700 1 -- 172.21.15.47:0/317088502 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f10004bb0 con 0x7f5f18107990 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f0effd700 1 -- 172.21.15.47:0/317088502 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10004020 con 0x7f5f18107990 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.894+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a5c50 con 0x7f5f18107990 2024-04-25T20:56:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.895+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f18109eb0 con 0x7f5f18107990 2024-04-25T20:56:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.897+0000 7f5f0effd700 1 -- 172.21.15.47:0/317088502 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5f1001b030 con 0x7f5f18107990 2024-04-25T20:56:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.897+0000 7f5f0effd700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f040a3c40 0x7f5f040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.897+0000 7f5f1deec700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f040a3c40 0x7f5f040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.897+0000 7f5f0effd700 1 -- 172.21.15.47:0/317088502 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f5f100cff30 con 0x7f5f18107990 2024-04-25T20:56:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.898+0000 7f5f1deec700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f040a3c40 0x7f5f040a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400b2e0 tx=0x7f5f1401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:55.905+0000 7f5f0effd700 1 -- 172.21.15.47:0/317088502 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f10022020 con 0x7f5f18107990 2024-04-25T20:56:56.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.115+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f18109890 con 0x7f5f18107990 2024-04-25T20:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.117+0000 7f5f0effd700 1 -- 172.21.15.47:0/317088502 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22160 (secure 0 0 0) 0x7f5f10010040 con 0x7f5f18107990 2024-04-25T20:56:56.119 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:56.119 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:56:53.086017+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:56:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f040a3c40 msgr2=0x7f5f040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f040a3c40 0x7f5f040a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400b2e0 tx=0x7f5f1401a040 comp rx=0 tx=0).stop 2024-04-25T20:56:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f18107990 msgr2=0x7f5f181a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f18107990 0x7f5f181a5030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000efc0 tx=0x7f5f1000c3a0 comp rx=0 tx=0).stop 2024-04-25T20:56:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 shutdown_connections 2024-04-25T20:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5f040a3c40 0x7f5f040a6100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f18107990 0x7f5f181a5030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f18101940 0x7f5f18199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 --2- 172.21.15.47:0/317088502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f18102300 0x7f5f1819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.119+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 >> 172.21.15.47:0/317088502 conn(0x7f5f180788f0 msgr2=0x7f5f181007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.120+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 shutdown_connections 2024-04-25T20:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:56.120+0000 7f5f20150700 1 -- 172.21.15.47:0/317088502 wait complete. 2024-04-25T20:56:56.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:56 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/317088502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:56:56.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:56 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/317088502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:56:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:56 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/317088502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:56:56.877 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:56:57.132 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:57.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:57 smithi047 ceph-mon[29195]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-04-25T20:56:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:57 smithi186 ceph-mon[32529]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-04-25T20:56:58.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:57 smithi092 ceph-mon[32524]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-04-25T20:56:58.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 -- 172.21.15.47:0/1089199601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 msgr2=0x7fd4bc10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/1089199601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 0x7fd4bc10f930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400b0a0 tx=0x7fd4b400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 -- 172.21.15.47:0/1089199601 shutdown_connections 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/1089199601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 0x7fd4bc10f930 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/1089199601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc108a60 0x7fd4bc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/1089199601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4bc06b1f0 0x7fd4bc06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 -- 172.21.15.47:0/1089199601 >> 172.21.15.47:0/1089199601 conn(0x7fd4bc0789e0 msgr2=0x7fd4bc078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 -- 172.21.15.47:0/1089199601 shutdown_connections 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.809+0000 7fd4c30e6700 1 -- 172.21.15.47:0/1089199601 wait complete. 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c30e6700 1 Processor -- start 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c30e6700 1 -- start start 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c30e6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc06b1f0 0x7fd4bc1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c30e6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 0x7fd4bc199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c30e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4bc108a60 0x7fd4bc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c30e6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc112ca0 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c30e6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc112b20 con 0x7fd4bc108a60 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c30e6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc112e20 con 0x7fd4bc06b1f0 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4bbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 0x7fd4bc199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4bbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 0x7fd4bc199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42830/0 (socket says 172.21.15.47:42830) 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4bbfff700 1 -- 172.21.15.47:0/4159881979 learned_addr learned my addr 172.21.15.47:0/4159881979 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c0e82700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc06b1f0 0x7fd4bc1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c1683700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4bc108a60 0x7fd4bc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4bbfff700 1 -- 172.21.15.47:0/4159881979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc06b1f0 msgr2=0x7fd4bc1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4bbfff700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc06b1f0 0x7fd4bc1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4bbfff700 1 -- 172.21.15.47:0/4159881979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4bc108a60 msgr2=0x7fd4bc19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4bbfff700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4bc108a60 0x7fd4bc19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4bbfff700 1 -- 172.21.15.47:0/4159881979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc19e0e0 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.810+0000 7fd4c1683700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4bc108a60 0x7fd4bc19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.811+0000 7fd4bbfff700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 0x7fd4bc199610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800d950 tx=0x7fd4a800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.811+0000 7fd4b9ffb700 1 -- 172.21.15.47:0/4159881979 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8016070 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.811+0000 7fd4b9ffb700 1 -- 172.21.15.47:0/4159881979 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a8010460 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.811+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4bc19e3d0 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.811+0000 7fd4b9ffb700 1 -- 172.21.15.47:0/4159881979 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800f620 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.811+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4bc1a5c50 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.812+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4bc069730 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.813+0000 7fd4b9ffb700 1 -- 172.21.15.47:0/4159881979 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd4a8019070 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.813+0000 7fd4b9ffb700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd4ac0a3c40 0x7fd4ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.814+0000 7fd4c0e82700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd4ac0a3c40 0x7fd4ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.814+0000 7fd4b9ffb700 1 -- 172.21.15.47:0/4159881979 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fd4a80d02b0 con 0x7fd4bc06bbb0 2024-04-25T20:56:58.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.814+0000 7fd4c0e82700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd4ac0a3c40 0x7fd4ac0a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009bd0 tx=0x7fd4b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:58.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:58.818+0000 7fd4b9ffb700 1 -- 172.21.15.47:0/4159881979 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4a809cde0 con 0x7fd4bc06bbb0 2024-04-25T20:56:59.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.026+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4bc02df80 con 0x7fd4bc06bbb0 2024-04-25T20:56:59.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.028+0000 7fd4b9ffb700 1 -- 172.21.15.47:0/4159881979 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22160 (secure 0 0 0) 0x7fd4a809c6e0 con 0x7fd4bc06bbb0 2024-04-25T20:56:59.029 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:56:59.029 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:56:53.086017+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd4ac0a3c40 msgr2=0x7fd4ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd4ac0a3c40 0x7fd4ac0a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009bd0 tx=0x7fd4b0009260 comp rx=0 tx=0).stop 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 msgr2=0x7fd4bc199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 0x7fd4bc199610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800d950 tx=0x7fd4a800dd10 comp rx=0 tx=0).stop 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 shutdown_connections 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fd4ac0a3c40 0x7fd4ac0a6100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4bc108a60 0x7fd4bc19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc06b1f0 0x7fd4bc1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 --2- 172.21.15.47:0/4159881979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4bc06bbb0 0x7fd4bc199610 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 >> 172.21.15.47:0/4159881979 conn(0x7fd4bc0789e0 msgr2=0x7fd4bc0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 shutdown_connections 2024-04-25T20:56:59.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:56:59.030+0000 7fd4c30e6700 1 -- 172.21.15.47:0/4159881979 wait complete. 2024-04-25T20:56:59.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-25T20:56:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:59.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:59 smithi186 ceph-mon[32529]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-25T20:56:59.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:56:59 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/4159881979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:00.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:59 smithi092 ceph-mon[32524]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-25T20:57:00.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:56:59 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/4159881979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:00.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:59 smithi047 ceph-mon[29195]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-25T20:57:00.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:56:59 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/4159881979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 -- 172.21.15.47:0/1235625298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101470 msgr2=0x7fb3781018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 --2- 172.21.15.47:0/1235625298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101470 0x7fb3781018f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb368009990 tx=0x7fb368009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 -- 172.21.15.47:0/1235625298 shutdown_connections 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 --2- 172.21.15.47:0/1235625298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb378101e30 0x7fb37810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 --2- 172.21.15.47:0/1235625298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378108c20 0x7fb378109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 --2- 172.21.15.47:0/1235625298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101470 0x7fb3781018f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 -- 172.21.15.47:0/1235625298 >> 172.21.15.47:0/1235625298 conn(0x7fb378076bb0 msgr2=0x7fb378076fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 -- 172.21.15.47:0/1235625298 shutdown_connections 2024-04-25T20:57:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 -- 172.21.15.47:0/1235625298 wait complete. 2024-04-25T20:57:01.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.443+0000 7fb38008e700 1 Processor -- start 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb38008e700 1 -- start start 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb38008e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378101470 0x7fb378102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb38008e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101e30 0x7fb378102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb38008e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb378108c20 0x7fb3781030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb38008e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb378112c50 con 0x7fb378101e30 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb38008e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb378112ad0 con 0x7fb378108c20 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb38008e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb378112dd0 con 0x7fb378101470 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101e30 0x7fb378102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101e30 0x7fb378102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42834/0 (socket says 172.21.15.47:42834) 2024-04-25T20:57:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 -- 172.21.15.47:0/2500478360 learned_addr learned my addr 172.21.15.47:0/2500478360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37de2a700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378101470 0x7fb378102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37e62b700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb378108c20 0x7fb3781030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 -- 172.21.15.47:0/2500478360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378101470 msgr2=0x7fb378102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378101470 0x7fb378102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 -- 172.21.15.47:0/2500478360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb378108c20 msgr2=0x7fb3781030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb378108c20 0x7fb3781030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 -- 172.21.15.47:0/2500478360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3781a75f0 con 0x7fb378101e30 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.444+0000 7fb37d629700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101e30 0x7fb378102b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb368000c00 tx=0x7fb3680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.445+0000 7fb36effd700 1 -- 172.21.15.47:0/2500478360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368027070 con 0x7fb378101e30 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.445+0000 7fb36effd700 1 -- 172.21.15.47:0/2500478360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb368003f20 con 0x7fb378101e30 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.445+0000 7fb36effd700 1 -- 172.21.15.47:0/2500478360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368022ca0 con 0x7fb378101e30 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.445+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781a7880 con 0x7fb378101e30 2024-04-25T20:57:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.445+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a7ce0 con 0x7fb378101e30 2024-04-25T20:57:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.446+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37810aff0 con 0x7fb378101e30 2024-04-25T20:57:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.447+0000 7fb36effd700 1 -- 172.21.15.47:0/2500478360 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb36802a070 con 0x7fb378101e30 2024-04-25T20:57:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.447+0000 7fb36effd700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb3640a3cc0 0x7fb3640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.448+0000 7fb37de2a700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb3640a3cc0 0x7fb3640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.448+0000 7fb36effd700 1 -- 172.21.15.47:0/2500478360 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fb3680d1450 con 0x7fb378101e30 2024-04-25T20:57:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.448+0000 7fb37de2a700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb3640a3cc0 0x7fb3640a6180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3740045b0 tx=0x7fb37400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.451+0000 7fb36effd700 1 -- 172.21.15.47:0/2500478360 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb368031070 con 0x7fb378101e30 2024-04-25T20:57:01.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:01.707+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb37810a9d0 con 0x7fb378101e30 2024-04-25T20:57:01.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:01 smithi186 ceph-mon[32529]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T20:57:02.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:01 smithi092 ceph-mon[32524]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T20:57:02.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:01 smithi047 ceph-mon[29195]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T20:57:02.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.689+0000 7fb36effd700 1 -- 172.21.15.47:0/2500478360 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fb368013070 con 0x7fb378101e30 2024-04-25T20:57:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb3640a3cc0 msgr2=0x7fb3640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb3640a3cc0 0x7fb3640a6180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3740045b0 tx=0x7fb37400b3c0 comp rx=0 tx=0).stop 2024-04-25T20:57:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101e30 msgr2=0x7fb378102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101e30 0x7fb378102b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb368000c00 tx=0x7fb3680047b0 comp rx=0 tx=0).stop 2024-04-25T20:57:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 shutdown_connections 2024-04-25T20:57:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb3640a3cc0 0x7fb3640a6180 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb378108c20 0x7fb3781030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378101470 0x7fb378102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 --2- 172.21.15.47:0/2500478360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb378101e30 0x7fb378102b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 >> 172.21.15.47:0/2500478360 conn(0x7fb378076bb0 msgr2=0x7fb378100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 shutdown_connections 2024-04-25T20:57:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:02.692+0000 7fb38008e700 1 -- 172.21.15.47:0/2500478360 wait complete. 2024-04-25T20:57:02.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:02 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2500478360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T20:57:03.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:02 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2500478360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T20:57:03.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:02 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2500478360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T20:57:03.335 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T20:57:03.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-04-25T20:57:03.589 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:03.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:03 smithi047 ceph-mon[29195]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T20:57:03.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:03 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2500478360' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T20:57:03.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:03 smithi047 ceph-mon[29195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:57:03.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:03 smithi186 ceph-mon[32529]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T20:57:03.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:03 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2500478360' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T20:57:03.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:03 smithi186 ceph-mon[32529]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:57:04.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:03 smithi092 ceph-mon[32524]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T20:57:04.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:03 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2500478360' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T20:57:04.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:03 smithi092 ceph-mon[32524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:57:05.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/1679834442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 msgr2=0x7f4e5410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:05.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/1679834442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 0x7f4e5410f930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00b0a0 tx=0x7f4e4c00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/1679834442 shutdown_connections 2024-04-25T20:57:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/1679834442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 0x7f4e5410f930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/1679834442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54108a60 0x7f4e54108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/1679834442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e5406b1f0 0x7f4e5406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/1679834442 >> 172.21.15.47:0/1679834442 conn(0x7f4e540789e0 msgr2=0x7f4e54078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/1679834442 shutdown_connections 2024-04-25T20:57:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.185+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/1679834442 wait complete. 2024-04-25T20:57:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5bb4a700 1 Processor -- start 2024-04-25T20:57:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5bb4a700 1 -- start start 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5bb4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e5406b1f0 0x7f4e54199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5bb4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 0x7f4e54199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5bb4a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54108a60 0x7f4e5419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5bb4a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112ca0 con 0x7f4e5406bbb0 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5bb4a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112b20 con 0x7f4e5406b1f0 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5bb4a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112e20 con 0x7f4e54108a60 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e590e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 0x7f4e54199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e590e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 0x7f4e54199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48320/0 (socket says 172.21.15.47:48320) 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e590e5700 1 -- 172.21.15.47:0/3156941229 learned_addr learned my addr 172.21.15.47:0/3156941229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e598e6700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e5406b1f0 0x7f4e54199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.186+0000 7f4e5a0e7700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54108a60 0x7f4e5419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e590e5700 1 -- 172.21.15.47:0/3156941229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54108a60 msgr2=0x7f4e5419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e590e5700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54108a60 0x7f4e5419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e590e5700 1 -- 172.21.15.47:0/3156941229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e5406b1f0 msgr2=0x7f4e54199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e590e5700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e5406b1f0 0x7f4e54199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e590e5700 1 -- 172.21.15.47:0/3156941229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5419e140 con 0x7f4e5406bbb0 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e590e5700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 0x7f4e54199670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400d950 tx=0x7f4e4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e4affd700 1 -- 172.21.15.47:0/3156941229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44016070 con 0x7f4e5406bbb0 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5419e430 con 0x7f4e5406bbb0 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e4affd700 1 -- 172.21.15.47:0/3156941229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e4400fd40 con 0x7f4e5406bbb0 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e4affd700 1 -- 172.21.15.47:0/3156941229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400e3b0 con 0x7f4e5406bbb0 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.187+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541a5cb0 con 0x7f4e5406bbb0 2024-04-25T20:57:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.188+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e54069730 con 0x7f4e5406bbb0 2024-04-25T20:57:05.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.189+0000 7f4e4affd700 1 -- 172.21.15.47:0/3156941229 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4e44019070 con 0x7f4e5406bbb0 2024-04-25T20:57:05.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.190+0000 7f4e4affd700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4e400a3c40 0x7f4e400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:05.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.190+0000 7f4e598e6700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4e400a3c40 0x7f4e400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:05.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.190+0000 7f4e4affd700 1 -- 172.21.15.47:0/3156941229 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f4e440cf790 con 0x7f4e5406bbb0 2024-04-25T20:57:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.190+0000 7f4e598e6700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4e400a3c40 0x7f4e400a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e50005950 tx=0x7f4e5000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:05.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.195+0000 7f4e4affd700 1 -- 172.21.15.47:0/3156941229 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e4409c2c0 con 0x7f4e5406bbb0 2024-04-25T20:57:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.452+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f4e5419a490 con 0x7f4e5406bbb0 2024-04-25T20:57:05.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.697+0000 7f4e4affd700 1 -- 172.21.15.47:0/3156941229 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f4e4409bbc0 con 0x7f4e5406bbb0 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4e400a3c40 msgr2=0x7f4e400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4e400a3c40 0x7f4e400a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e50005950 tx=0x7f4e5000b410 comp rx=0 tx=0).stop 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 msgr2=0x7f4e54199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 0x7f4e54199670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400d950 tx=0x7f4e4400dc60 comp rx=0 tx=0).stop 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 shutdown_connections 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4e400a3c40 0x7f4e400a6100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e54108a60 0x7f4e5419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e5406b1f0 0x7f4e54199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 --2- 172.21.15.47:0/3156941229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e5406bbb0 0x7f4e54199670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 >> 172.21.15.47:0/3156941229 conn(0x7f4e540789e0 msgr2=0x7f4e540ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 shutdown_connections 2024-04-25T20:57:05.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:05.700+0000 7f4e5bb4a700 1 -- 172.21.15.47:0/3156941229 wait complete. 2024-04-25T20:57:05.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:05 smithi047 ceph-mon[29195]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T20:57:05.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:05 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3156941229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T20:57:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:05 smithi186 ceph-mon[32529]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T20:57:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:05 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3156941229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T20:57:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:05 smithi092 ceph-mon[32524]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T20:57:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:05 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3156941229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T20:57:06.308 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-25T20:57:06.566 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:06.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:06 smithi047 ceph-mon[29195]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T20:57:06.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:06 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3156941229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T20:57:06.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:06 smithi047 ceph-mon[29195]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T20:57:06.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:06 smithi047 ceph-mon[29195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:57:06.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:06 smithi047 ceph-mon[29195]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T20:57:06.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:06 smithi047 ceph-mon[29195]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T20:57:06.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:06 smithi186 ceph-mon[32529]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T20:57:06.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:06 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3156941229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T20:57:06.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:06 smithi186 ceph-mon[32529]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T20:57:06.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:06 smithi186 ceph-mon[32529]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:57:06.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:06 smithi186 ceph-mon[32529]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T20:57:06.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:06 smithi186 ceph-mon[32529]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T20:57:07.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:06 smithi092 ceph-mon[32524]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T20:57:07.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:06 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3156941229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T20:57:07.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:06 smithi092 ceph-mon[32524]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T20:57:07.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:06 smithi092 ceph-mon[32524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:57:07.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:06 smithi092 ceph-mon[32524]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T20:57:07.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:06 smithi092 ceph-mon[32524]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T20:57:07.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:07 smithi186 ceph-mon[32529]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T20:57:07.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:07 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] up:active 2024-04-25T20:57:07.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:07 smithi186 ceph-mon[32529]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T20:57:07.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:07 smithi186 ceph-mon[32529]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T20:57:07.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:07 smithi186 ceph-mon[32529]: Cluster is now healthy 2024-04-25T20:57:07.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:07 smithi186 ceph-mon[32529]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T20:57:07.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:07 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-25T20:57:07.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:07 smithi186 ceph-mon[32529]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-25T20:57:07.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:07 smithi047 ceph-mon[29195]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T20:57:07.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:07 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] up:active 2024-04-25T20:57:07.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:07 smithi047 ceph-mon[29195]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T20:57:07.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:07 smithi047 ceph-mon[29195]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T20:57:07.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:07 smithi047 ceph-mon[29195]: Cluster is now healthy 2024-04-25T20:57:07.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:07 smithi047 ceph-mon[29195]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T20:57:07.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:07 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-25T20:57:07.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:07 smithi047 ceph-mon[29195]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-25T20:57:08.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:07 smithi092 ceph-mon[32524]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T20:57:08.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:07 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] up:active 2024-04-25T20:57:08.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:07 smithi092 ceph-mon[32524]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T20:57:08.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:07 smithi092 ceph-mon[32524]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T20:57:08.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:07 smithi092 ceph-mon[32524]: Cluster is now healthy 2024-04-25T20:57:08.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:07 smithi092 ceph-mon[32524]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T20:57:08.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:07 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-25T20:57:08.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:07 smithi092 ceph-mon[32524]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-25T20:57:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 -- 172.21.15.47:0/265956138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06b0f0 msgr2=0x7f2f0c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/265956138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06b0f0 0x7f2f0c10f900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b0a0 tx=0x7f2f0400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 -- 172.21.15.47:0/265956138 shutdown_connections 2024-04-25T20:57:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/265956138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06b0f0 0x7f2f0c10f900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/265956138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f0c108a30 0x7f2f0c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/265956138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f0c06a750 0x7f2f0c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 -- 172.21.15.47:0/265956138 >> 172.21.15.47:0/265956138 conn(0x7f2f0c0788f0 msgr2=0x7f2f0c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 -- 172.21.15.47:0/265956138 shutdown_connections 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.383+0000 7f2f11bf8700 1 -- 172.21.15.47:0/265956138 wait complete. 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.384+0000 7f2f11bf8700 1 Processor -- start 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.384+0000 7f2f11bf8700 1 -- start start 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f11bf8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06a750 0x7f2f0c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f11bf8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f0c06b0f0 0x7f2f0c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f11bf8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f0c108a30 0x7f2f0c19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f11bf8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c112bd0 con 0x7f2f0c06a750 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06a750 0x7f2f0c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f11bf8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c112a50 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f11bf8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c112d50 con 0x7f2f0c108a30 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06a750 0x7f2f0c1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48340/0 (socket says 172.21.15.47:48340) 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f0c06b0f0 0x7f2f0c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0b7fe700 1 -- 172.21.15.47:0/1642314340 learned_addr learned my addr 172.21.15.47:0/1642314340 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0bfff700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f0c108a30 0x7f2f0c19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0affd700 1 -- 172.21.15.47:0/1642314340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f0c108a30 msgr2=0x7f2f0c19d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0affd700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f0c108a30 0x7f2f0c19d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0affd700 1 -- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06a750 msgr2=0x7f2f0c1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0affd700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06a750 0x7f2f0c1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0affd700 1 -- 172.21.15.47:0/1642314340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c19e0a0 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0b7fe700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06a750 0x7f2f0c1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.385+0000 7f2f0affd700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f0c06b0f0 0x7f2f0c199620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c970 tx=0x7f2ef400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.386+0000 7f2f08ff9700 1 -- 172.21.15.47:0/1642314340 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4014070 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.386+0000 7f2f08ff9700 1 -- 172.21.15.47:0/1642314340 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef4004440 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.386+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c19e390 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.386+0000 7f2f08ff9700 1 -- 172.21.15.47:0/1642314340 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4007a30 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.386+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c101940 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.387+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0c069730 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.388+0000 7f2f08ff9700 1 -- 172.21.15.47:0/1642314340 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2ef4017070 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.388+0000 7f2f08ff9700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2ef80a3c40 0x7f2ef80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.389+0000 7f2f0b7fe700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2ef80a3c40 0x7f2ef80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:08.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.389+0000 7f2f08ff9700 1 -- 172.21.15.47:0/1642314340 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f2ef40cfc10 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.389+0000 7f2f0b7fe700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2ef80a3c40 0x7f2ef80a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009770 tx=0x7f2efc012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:08.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.398+0000 7f2f08ff9700 1 -- 172.21.15.47:0/1642314340 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ef401e030 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.654+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2f0c02df80 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.722+0000 7f2f08ff9700 1 -- 172.21.15.47:0/1642314340 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f2ef409c6a0 con 0x7f2f0c06b0f0 2024-04-25T20:57:08.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2ef80a3c40 msgr2=0x7f2ef80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2ef80a3c40 0x7f2ef80a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009770 tx=0x7f2efc012040 comp rx=0 tx=0).stop 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f0c06b0f0 msgr2=0x7f2f0c199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f0c06b0f0 0x7f2f0c199620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c970 tx=0x7f2ef400cc80 comp rx=0 tx=0).stop 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 shutdown_connections 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2ef80a3c40 0x7f2ef80a6100 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f0c06a750 0x7f2f0c1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f0c108a30 0x7f2f0c19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 --2- 172.21.15.47:0/1642314340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f0c06b0f0 0x7f2f0c199620 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 >> 172.21.15.47:0/1642314340 conn(0x7f2f0c0788f0 msgr2=0x7f2f0c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 shutdown_connections 2024-04-25T20:57:08.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:08.725+0000 7f2f11bf8700 1 -- 172.21.15.47:0/1642314340 wait complete. 2024-04-25T20:57:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:08 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] up:active 2024-04-25T20:57:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:08 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-25T20:57:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:08 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1642314340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:57:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:08 smithi047 ceph-mon[29195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:57:09.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:08 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] up:active 2024-04-25T20:57:09.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:08 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-25T20:57:09.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:08 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1642314340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:57:09.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:08 smithi092 ceph-mon[32524]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:57:09.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:08 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] up:active 2024-04-25T20:57:09.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:08 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-25T20:57:09.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:08 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1642314340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:57:09.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:08 smithi186 ceph-mon[32529]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:57:09.506 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-25T20:57:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:10.022 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:09 smithi047 ceph-mon[29195]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-25T20:57:10.022 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:09 smithi047 ceph-mon[29195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T20:57:10.022 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:09 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-25T20:57:10.022 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:09 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:10.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:09 smithi092 ceph-mon[32524]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-25T20:57:10.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:09 smithi092 ceph-mon[32524]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T20:57:10.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:09 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-25T20:57:10.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:09 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:10.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:09 smithi186 ceph-mon[32529]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-25T20:57:10.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:09 smithi186 ceph-mon[32529]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T20:57:10.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:09 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-25T20:57:10.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:09 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:11.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2161133734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4108a50 msgr2=0x7f4ce4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:11.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2161133734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4108a50 0x7f4ce4108e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009990 tx=0x7f4ce0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:11.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2161133734 shutdown_connections 2024-04-25T20:57:11.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2161133734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4108a50 0x7f4ce4108e30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:11.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2161133734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ce4103450 0x7f4ce410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:11.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2161133734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ce4102a90 0x7f4ce4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:11.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2161133734 >> 172.21.15.47:0/2161133734 conn(0x7f4ce4079ba0 msgr2=0x7f4ce4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:11.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2161133734 shutdown_connections 2024-04-25T20:57:11.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.549+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2161133734 wait complete. 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cec3f2700 1 Processor -- start 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cec3f2700 1 -- start start 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cec3f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4102a90 0x7f4ce4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cec3f2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ce4103450 0x7f4ce4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cec3f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ce4108a50 0x7f4ce40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cec3f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4112c00 con 0x7f4ce4102a90 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cec3f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4112a80 con 0x7f4ce4108a50 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cec3f2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4112d80 con 0x7f4ce4103450 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4102a90 0x7f4ce4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4102a90 0x7f4ce4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48364/0 (socket says 172.21.15.47:48364) 2024-04-25T20:57:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 -- 172.21.15.47:0/2310458961 learned_addr learned my addr 172.21.15.47:0/2310458961 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea98f700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ce4108a50 0x7f4ce40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4ce998d700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ce4103450 0x7f4ce4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 -- 172.21.15.47:0/2310458961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ce4103450 msgr2=0x7f4ce4075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ce4103450 0x7f4ce4075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 -- 172.21.15.47:0/2310458961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ce4108a50 msgr2=0x7f4ce40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ce4108a50 0x7f4ce40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 -- 172.21.15.47:0/2310458961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce41a76c0 con 0x7f4ce4102a90 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.550+0000 7f4cea18e700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4102a90 0x7f4ce4074d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0000c00 tx=0x7f4ce00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.551+0000 7f4cdb7fe700 1 -- 172.21.15.47:0/2310458961 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0027070 con 0x7f4ce4102a90 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.551+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce41a7950 con 0x7f4ce4102a90 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.551+0000 7f4cdb7fe700 1 -- 172.21.15.47:0/2310458961 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0003f20 con 0x7f4ce4102a90 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.551+0000 7f4cdb7fe700 1 -- 172.21.15.47:0/2310458961 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0022cf0 con 0x7f4ce4102a90 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.551+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce41a7e30 con 0x7f4ce4102a90 2024-04-25T20:57:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.552+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce4069730 con 0x7f4ce4102a90 2024-04-25T20:57:11.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.553+0000 7f4cdb7fe700 1 -- 172.21.15.47:0/2310458961 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4ce002a070 con 0x7f4ce4102a90 2024-04-25T20:57:11.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.554+0000 7f4cdb7fe700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cd00a3c40 0x7f4cd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:11.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.554+0000 7f4ce998d700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cd00a3c40 0x7f4cd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:11.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.554+0000 7f4cdb7fe700 1 -- 172.21.15.47:0/2310458961 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f4ce00d14a0 con 0x7f4ce4102a90 2024-04-25T20:57:11.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.554+0000 7f4ce998d700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cd00a3c40 0x7f4cd00a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4005950 tx=0x7f4cd40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:11.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.557+0000 7f4cdb7fe700 1 -- 172.21.15.47:0/2310458961 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ce0013070 con 0x7f4ce4102a90 2024-04-25T20:57:11.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:11.812+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f4ce402e1a0 con 0x7f4ce4102a90 2024-04-25T20:57:12.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:11 smithi092 ceph-mon[32524]: pgmap v301: 97 pgs: 97 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T20:57:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:11 smithi047 ceph-mon[29195]: pgmap v301: 97 pgs: 97 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T20:57:12.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:11 smithi186 ceph-mon[32529]: pgmap v301: 97 pgs: 97 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T20:57:12.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.731+0000 7f4cdb7fe700 1 -- 172.21.15.47:0/2310458961 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f4ce009dfc0 con 0x7f4ce4102a90 2024-04-25T20:57:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cd00a3c40 msgr2=0x7f4cd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cd00a3c40 0x7f4cd00a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4005950 tx=0x7f4cd40058e0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4102a90 msgr2=0x7f4ce4074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4102a90 0x7f4ce4074d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0000c00 tx=0x7f4ce00047b0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 shutdown_connections 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cd00a3c40 0x7f4cd00a6100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce4102a90 0x7f4ce4074d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ce4108a50 0x7f4ce40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 --2- 172.21.15.47:0/2310458961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ce4103450 0x7f4ce4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.734+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 >> 172.21.15.47:0/2310458961 conn(0x7f4ce4079ba0 msgr2=0x7f4ce4101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.735+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 shutdown_connections 2024-04-25T20:57:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:12.735+0000 7f4cec3f2700 1 -- 172.21.15.47:0/2310458961 wait complete. 2024-04-25T20:57:13.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:12 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2310458961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T20:57:13.053 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:12 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2310458961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T20:57:13.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:12 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2310458961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T20:57:13.413 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T20:57:13.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-25T20:57:13.669 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:13 smithi092 ceph-mon[32524]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T20:57:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:13 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2310458961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T20:57:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:13 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:14.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:13 smithi047 ceph-mon[29195]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T20:57:14.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:13 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2310458961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T20:57:14.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:13 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:14.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:13 smithi186 ceph-mon[32529]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T20:57:14.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:13 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2310458961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T20:57:14.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:13 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:15.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 -- 172.21.15.47:0/2912873441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8102300 msgr2=0x7f20a810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 --2- 172.21.15.47:0/2912873441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8102300 0x7f20a810e7f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009990 tx=0x7f20a0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 -- 172.21.15.47:0/2912873441 shutdown_connections 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 --2- 172.21.15.47:0/2912873441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8102300 0x7f20a810e7f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 --2- 172.21.15.47:0/2912873441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8107990 0x7f20a8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 --2- 172.21.15.47:0/2912873441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20a8101940 0x7f20a8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 -- 172.21.15.47:0/2912873441 >> 172.21.15.47:0/2912873441 conn(0x7f20a80788f0 msgr2=0x7f20a8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 -- 172.21.15.47:0/2912873441 shutdown_connections 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.377+0000 7f20b01ad700 1 -- 172.21.15.47:0/2912873441 wait complete. 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20b01ad700 1 Processor -- start 2024-04-25T20:57:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20b01ad700 1 -- start start 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20b01ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8101940 0x7f20a81034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20b01ad700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20a8102300 0x7f20a8103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20b01ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8107990 0x7f20a8103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20b01ad700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f20a8111ae0 con 0x7f20a8107990 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20b01ad700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f20a8111960 con 0x7f20a8101940 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20b01ad700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f20a8111c60 con 0x7f20a8102300 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20ae74a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8107990 0x7f20a8103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20ae74a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8107990 0x7f20a8103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55486/0 (socket says 172.21.15.47:55486) 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20ae74a700 1 -- 172.21.15.47:0/3507579840 learned_addr learned my addr 172.21.15.47:0/3507579840 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20ad748700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20a8102300 0x7f20a8103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.378+0000 7f20adf49700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8101940 0x7f20a81034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:15.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20ae74a700 1 -- 172.21.15.47:0/3507579840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20a8102300 msgr2=0x7f20a8103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20ae74a700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20a8102300 0x7f20a8103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20ae74a700 1 -- 172.21.15.47:0/3507579840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8101940 msgr2=0x7f20a81034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20ae74a700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8101940 0x7f20a81034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20ae74a700 1 -- 172.21.15.47:0/3507579840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a81a7580 con 0x7f20a8107990 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20ae74a700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8107990 0x7f20a8103f70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f20a00047b0 tx=0x7f20a0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20a6ffd700 1 -- 172.21.15.47:0/3507579840 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0027070 con 0x7f20a8107990 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a81a7810 con 0x7f20a8107990 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20a6ffd700 1 -- 172.21.15.47:0/3507579840 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a0003f20 con 0x7f20a8107990 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a81a7cc0 con 0x7f20a8107990 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.379+0000 7f20a6ffd700 1 -- 172.21.15.47:0/3507579840 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0022bb0 con 0x7f20a8107990 2024-04-25T20:57:15.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.380+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a8069730 con 0x7f20a8107990 2024-04-25T20:57:15.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.382+0000 7f20a6ffd700 1 -- 172.21.15.47:0/3507579840 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f20a002a070 con 0x7f20a8107990 2024-04-25T20:57:15.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.382+0000 7f20a6ffd700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f20900a3c40 0x7f20900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:15.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.383+0000 7f20adf49700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f20900a3c40 0x7f20900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:15.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.383+0000 7f20a6ffd700 1 -- 172.21.15.47:0/3507579840 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f20a00d1940 con 0x7f20a8107990 2024-04-25T20:57:15.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.383+0000 7f20adf49700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f20900a3c40 0x7f20900a6100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2098009c90 tx=0x7f2098009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:15.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.386+0000 7f20a6ffd700 1 -- 172.21.15.47:0/3507579840 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20a0031070 con 0x7f20a8107990 2024-04-25T20:57:15.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.623+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f20a80ff7b0 con 0x7f20900a3c40 2024-04-25T20:57:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.651+0000 7f20a6ffd700 1 -- 172.21.15.47:0/3507579840 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20a80ff7b0 con 0x7f20900a3c40 2024-04-25T20:57:15.652 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:57:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:57:15.628+0000 7f97eb533700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:15.653 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:57:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:57:15.628+0000 7f97eb533700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:15.653 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:57:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:57:15.628+0000 7f97eb533700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:15.653 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:57:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:57:15.628+0000 7f97eb533700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:15.653 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:57:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:57:15.628+0000 7f97eb533700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f20900a3c40 msgr2=0x7f20900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f20900a3c40 0x7f20900a6100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2098009c90 tx=0x7f2098009040 comp rx=0 tx=0).stop 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8107990 msgr2=0x7f20a8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8107990 0x7f20a8103f70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f20a00047b0 tx=0x7f20a0005e80 comp rx=0 tx=0).stop 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 shutdown_connections 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f20900a3c40 0x7f20900a6100 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a8107990 0x7f20a8103f70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8101940 0x7f20a81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 --2- 172.21.15.47:0/3507579840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20a8102300 0x7f20a8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 >> 172.21.15.47:0/3507579840 conn(0x7f20a80788f0 msgr2=0x7f20a8105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 shutdown_connections 2024-04-25T20:57:15.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:15.654+0000 7f20b01ad700 1 -- 172.21.15.47:0/3507579840 wait complete. 2024-04-25T20:57:16.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:15 smithi092 ceph-mon[32524]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:57:16.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:15 smithi047 ceph-mon[29195]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:57:16.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:15 smithi186 ceph-mon[32529]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:57:16.264 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T20:57:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:16.972 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:16 smithi047 ceph-mon[29195]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:17.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:16 smithi092 ceph-mon[32524]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:17.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:16 smithi186 ceph-mon[32529]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:17.938 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:17 smithi047 ceph-mon[29195]: pgmap v304: 97 pgs: 97 active+clean; 588 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T20:57:17.938 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:17 smithi047 ceph-mon[29195]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T20:57:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:17 smithi092 ceph-mon[32524]: pgmap v304: 97 pgs: 97 active+clean; 588 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T20:57:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:17 smithi092 ceph-mon[32524]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T20:57:18.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 -- 172.21.15.47:0/2246441719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 msgr2=0x7f9964101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 --2- 172.21.15.47:0/2246441719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 0x7f9964101880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9954009990 tx=0x7f9954009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 -- 172.21.15.47:0/2246441719 shutdown_connections 2024-04-25T20:57:18.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 --2- 172.21.15.47:0/2246441719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9964101dc0 0x7f996410f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 --2- 172.21.15.47:0/2246441719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964107b20 0x7f9964107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 --2- 172.21.15.47:0/2246441719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 0x7f9964101880 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 -- 172.21.15.47:0/2246441719 >> 172.21.15.47:0/2246441719 conn(0x7f9964076c60 msgr2=0x7f9964077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 -- 172.21.15.47:0/2246441719 shutdown_connections 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 -- 172.21.15.47:0/2246441719 wait complete. 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 Processor -- start 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.198+0000 7f996b2e9700 1 -- start start 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f996b2e9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 0x7f9964199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f996b2e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9964101dc0 0x7f9964199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f996b2e9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964107b20 0x7f996419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f996b2e9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9964112cb0 con 0x7f9964101400 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f996b2e9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9964112b30 con 0x7f9964101dc0 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f996b2e9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9964112e30 con 0x7f9964107b20 2024-04-25T20:57:18.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 0x7f9964199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969886700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964107b20 0x7f996419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 0x7f9964199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55536/0 (socket says 172.21.15.47:55536) 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969886700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964107b20 0x7f996419d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:49794/0 (socket says 172.21.15.47:49794) 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 -- 172.21.15.47:0/689999545 learned_addr learned my addr 172.21.15.47:0/689999545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9968884700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9964101dc0 0x7f9964199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 -- 172.21.15.47:0/689999545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964107b20 msgr2=0x7f996419d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964107b20 0x7f996419d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 -- 172.21.15.47:0/689999545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9964101dc0 msgr2=0x7f9964199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9964101dc0 0x7f9964199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 -- 172.21.15.47:0/689999545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996419e050 con 0x7f9964101400 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f9969085700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 0x7f9964199040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f996000d980 tx=0x7f996000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f995a7fc700 1 -- 172.21.15.47:0/689999545 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960017030 con 0x7f9964101400 2024-04-25T20:57:18.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f995a7fc700 1 -- 172.21.15.47:0/689999545 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99600044d0 con 0x7f9964101400 2024-04-25T20:57:18.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996419e340 con 0x7f9964101400 2024-04-25T20:57:18.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.199+0000 7f995a7fc700 1 -- 172.21.15.47:0/689999545 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960003970 con 0x7f9964101400 2024-04-25T20:57:18.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.200+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99641a5bc0 con 0x7f9964101400 2024-04-25T20:57:18.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.200+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9964069730 con 0x7f9964101400 2024-04-25T20:57:18.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.202+0000 7f995a7fc700 1 -- 172.21.15.47:0/689999545 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f996001a070 con 0x7f9964101400 2024-04-25T20:57:18.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.203+0000 7f995a7fc700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99500a3d50 0x7f99500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:18.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.203+0000 7f9968884700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99500a3d50 0x7f99500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:18.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.203+0000 7f995a7fc700 1 -- 172.21.15.47:0/689999545 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f99600cff40 con 0x7f9964101400 2024-04-25T20:57:18.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.203+0000 7f9968884700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99500a3d50 0x7f99500a6210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f995400b5c0 tx=0x7f9954005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:18.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.207+0000 7f995a7fc700 1 -- 172.21.15.47:0/689999545 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f996009c9f0 con 0x7f9964101400 2024-04-25T20:57:18.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:17 smithi186 ceph-mon[32529]: pgmap v304: 97 pgs: 97 active+clean; 588 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T20:57:18.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:17 smithi186 ceph-mon[32529]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T20:57:18.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.446+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9964062e80 con 0x7f99500a3d50 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.474+0000 7f995a7fc700 1 -- 172.21.15.47:0/689999545 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9964062e80 con 0x7f99500a3d50 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99500a3d50 msgr2=0x7f99500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99500a3d50 0x7f99500a6210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f995400b5c0 tx=0x7f9954005960 comp rx=0 tx=0).stop 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 msgr2=0x7f9964199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 0x7f9964199040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f996000d980 tx=0x7f996000dc90 comp rx=0 tx=0).stop 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 shutdown_connections 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f99500a3d50 0x7f99500a6210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9964101400 0x7f9964199040 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964107b20 0x7f996419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 --2- 172.21.15.47:0/689999545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9964101dc0 0x7f9964199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 >> 172.21.15.47:0/689999545 conn(0x7f9964076c60 msgr2=0x7f99640fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 shutdown_connections 2024-04-25T20:57:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:18.476+0000 7f996b2e9700 1 -- 172.21.15.47:0/689999545 wait complete. 2024-04-25T20:57:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:18 smithi092 ceph-mon[32524]: pgmap v305: 97 pgs: 97 active+clean; 588 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T20:57:19.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:18 smithi092 ceph-mon[32524]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:19.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:18 smithi047 ceph-mon[29195]: pgmap v305: 97 pgs: 97 active+clean; 588 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T20:57:19.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:18 smithi047 ceph-mon[29195]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:19.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T20:57:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:18 smithi186 ceph-mon[32529]: pgmap v305: 97 pgs: 97 active+clean; 588 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T20:57:19.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:18 smithi186 ceph-mon[32529]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:20 smithi047 ceph-mon[29195]: pgmap v306: 97 pgs: 97 active+clean; 589 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 -- 172.21.15.47:0/3280084176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 msgr2=0x7fbd2c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/3280084176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 0x7fbd2c101de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009990 tx=0x7fbd20009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 -- 172.21.15.47:0/3280084176 shutdown_connections 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/3280084176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c102320 0x7fbd2c10e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/3280084176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd2c107960 0x7fbd2c107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/3280084176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 0x7fbd2c101de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 -- 172.21.15.47:0/3280084176 >> 172.21.15.47:0/3280084176 conn(0x7fbd2c0789e0 msgr2=0x7fbd2c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 -- 172.21.15.47:0/3280084176 shutdown_connections 2024-04-25T20:57:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.156+0000 7fbd31f1b700 1 -- 172.21.15.47:0/3280084176 wait complete. 2024-04-25T20:57:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd31f1b700 1 Processor -- start 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd31f1b700 1 -- start start 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd31f1b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 0x7fbd2c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd31f1b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd2c102320 0x7fbd2c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd31f1b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c107960 0x7fbd2c19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd31f1b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112c90 con 0x7fbd2c101960 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd31f1b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112b10 con 0x7fbd2c102320 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd31f1b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112e10 con 0x7fbd2c107960 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 0x7fbd2c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 0x7fbd2c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55566/0 (socket says 172.21.15.47:55566) 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd2b7fe700 1 -- 172.21.15.47:0/690741415 learned_addr learned my addr 172.21.15.47:0/690741415 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd2bfff700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c107960 0x7fbd2c19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.157+0000 7fbd2affd700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd2c102320 0x7fbd2c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd2b7fe700 1 -- 172.21.15.47:0/690741415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c107960 msgr2=0x7fbd2c19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd2b7fe700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c107960 0x7fbd2c19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd2b7fe700 1 -- 172.21.15.47:0/690741415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd2c102320 msgr2=0x7fbd2c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd2b7fe700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd2c102320 0x7fbd2c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd2b7fe700 1 -- 172.21.15.47:0/690741415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c19e250 con 0x7fbd2c101960 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd2affd700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd2c102320 0x7fbd2c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd2bfff700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c107960 0x7fbd2c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd2b7fe700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 0x7fbd2c199240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00d980 tx=0x7fbd1c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd28ff9700 1 -- 172.21.15.47:0/690741415 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c016070 con 0x7fbd2c101960 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd28ff9700 1 -- 172.21.15.47:0/690741415 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd1c0044d0 con 0x7fbd2c101960 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c19e540 con 0x7fbd2c101960 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd28ff9700 1 -- 172.21.15.47:0/690741415 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c004d00 con 0x7fbd2c101960 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.158+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c1a5dc0 con 0x7fbd2c101960 2024-04-25T20:57:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.159+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2c069730 con 0x7fbd2c101960 2024-04-25T20:57:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.160+0000 7fbd28ff9700 1 -- 172.21.15.47:0/690741415 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd1c004020 con 0x7fbd2c101960 2024-04-25T20:57:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.161+0000 7fbd28ff9700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbd140a3cd0 0x7fbd140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.161+0000 7fbd2affd700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbd140a3cd0 0x7fbd140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.161+0000 7fbd28ff9700 1 -- 172.21.15.47:0/690741415 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fbd1c0d08f0 con 0x7fbd2c101960 2024-04-25T20:57:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.161+0000 7fbd2affd700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbd140a3cd0 0x7fbd140a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd20005b40 tx=0x7fbd20005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.165+0000 7fbd28ff9700 1 -- 172.21.15.47:0/690741415 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd1c01f020 con 0x7fbd2c101960 2024-04-25T20:57:21.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:20 smithi186 ceph-mon[32529]: pgmap v306: 97 pgs: 97 active+clean; 589 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-25T20:57:21.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:20 smithi092 ceph-mon[32524]: pgmap v306: 97 pgs: 97 active+clean; 589 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-25T20:57:21.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.404+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fbd2c19a5a0 con 0x7fbd140a3cd0 2024-04-25T20:57:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.410+0000 7fbd28ff9700 1 -- 172.21.15.47:0/690741415 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbd2c19a5a0 con 0x7fbd140a3cd0 2024-04-25T20:57:21.411 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_0/614db76e-1650-4ba3-bbe2-354a2dc8c12a 2024-04-25T20:57:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbd140a3cd0 msgr2=0x7fbd140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbd140a3cd0 0x7fbd140a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd20005b40 tx=0x7fbd20005a90 comp rx=0 tx=0).stop 2024-04-25T20:57:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 msgr2=0x7fbd2c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 0x7fbd2c199240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00d980 tx=0x7fbd1c00dd40 comp rx=0 tx=0).stop 2024-04-25T20:57:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 shutdown_connections 2024-04-25T20:57:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbd140a3cd0 0x7fbd140a6190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd2c101960 0x7fbd2c199240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c107960 0x7fbd2c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 --2- 172.21.15.47:0/690741415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd2c102320 0x7fbd2c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 >> 172.21.15.47:0/690741415 conn(0x7fbd2c0789e0 msgr2=0x7fbd2c104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 shutdown_connections 2024-04-25T20:57:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:21.412+0000 7fbd31f1b700 1 -- 172.21.15.47:0/690741415 wait complete. 2024-04-25T20:57:22.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T20:57:22.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:21 smithi186 ceph-mon[32529]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:22.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:21 smithi092 ceph-mon[32524]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:22.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:21 smithi047 ceph-mon[29195]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:23.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:22 smithi047 ceph-mon[29195]: pgmap v307: 97 pgs: 97 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-25T20:57:23.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:22 smithi092 ceph-mon[32524]: pgmap v307: 97 pgs: 97 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-25T20:57:23.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:22 smithi186 ceph-mon[32529]: pgmap v307: 97 pgs: 97 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-25T20:57:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 -- 172.21.15.47:0/784716258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006bbb0 msgr2=0x7fe67010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 --2- 172.21.15.47:0/784716258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006bbb0 0x7fe67010f930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe66800b0a0 tx=0x7fe66800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 -- 172.21.15.47:0/784716258 shutdown_connections 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 --2- 172.21.15.47:0/784716258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006bbb0 0x7fe67010f930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 --2- 172.21.15.47:0/784716258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe670108a60 0x7fe670108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 --2- 172.21.15.47:0/784716258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe67006b1f0 0x7fe67006b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 -- 172.21.15.47:0/784716258 >> 172.21.15.47:0/784716258 conn(0x7fe6700789e0 msgr2=0x7fe670078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 -- 172.21.15.47:0/784716258 shutdown_connections 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 -- 172.21.15.47:0/784716258 wait complete. 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 Processor -- start 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.085+0000 7fe67628e700 1 -- start start 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe67628e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006b1f0 0x7fe6701019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe67628e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67006bbb0 0x7fe670101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe67628e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe670108a60 0x7fe670102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe67628e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe670112ca0 con 0x7fe67006b1f0 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe67628e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe670112b20 con 0x7fe670108a60 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe67628e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe670112e20 con 0x7fe67006bbb0 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe66ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006b1f0 0x7fe6701019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe66ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006b1f0 0x7fe6701019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50296/0 (socket says 172.21.15.47:50296) 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe66ffff700 1 -- 172.21.15.47:0/3703188497 learned_addr learned my addr 172.21.15.47:0/3703188497 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe66f7fe700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67006bbb0 0x7fe670101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe67482b700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe670108a60 0x7fe670102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe66ffff700 1 -- 172.21.15.47:0/3703188497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67006bbb0 msgr2=0x7fe670101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe66ffff700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67006bbb0 0x7fe670101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe66ffff700 1 -- 172.21.15.47:0/3703188497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe670108a60 msgr2=0x7fe670102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.086+0000 7fe66ffff700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe670108a60 0x7fe670102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.087+0000 7fe66ffff700 1 -- 172.21.15.47:0/3703188497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6701a7710 con 0x7fe67006b1f0 2024-04-25T20:57:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.087+0000 7fe66ffff700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006b1f0 0x7fe6701019b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe66000d980 tx=0x7fe66000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.087+0000 7fe66d7fa700 1 -- 172.21.15.47:0/3703188497 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660017070 con 0x7fe67006b1f0 2024-04-25T20:57:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.087+0000 7fe66d7fa700 1 -- 172.21.15.47:0/3703188497 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe660005020 con 0x7fe67006b1f0 2024-04-25T20:57:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.087+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6701a7a00 con 0x7fe67006b1f0 2024-04-25T20:57:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.087+0000 7fe66d7fa700 1 -- 172.21.15.47:0/3703188497 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66000fa10 con 0x7fe67006b1f0 2024-04-25T20:57:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.087+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6701a7ef0 con 0x7fe67006b1f0 2024-04-25T20:57:24.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.088+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe670069730 con 0x7fe67006b1f0 2024-04-25T20:57:24.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.089+0000 7fe66d7fa700 1 -- 172.21.15.47:0/3703188497 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe660004a30 con 0x7fe67006b1f0 2024-04-25T20:57:24.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.090+0000 7fe66d7fa700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6580a3cd0 0x7fe6580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.090+0000 7fe66f7fe700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6580a3cd0 0x7fe6580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.090+0000 7fe66d7fa700 1 -- 172.21.15.47:0/3703188497 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fe6600d0360 con 0x7fe67006b1f0 2024-04-25T20:57:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.090+0000 7fe66f7fe700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6580a3cd0 0x7fe6580a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe670078fc0 tx=0x7fe664009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.094+0000 7fe66d7fa700 1 -- 172.21.15.47:0/3703188497 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe660020020 con 0x7fe67006b1f0 2024-04-25T20:57:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.338+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe670062e80 con 0x7fe6580a3cd0 2024-04-25T20:57:24.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.366+0000 7fe66d7fa700 1 -- 172.21.15.47:0/3703188497 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe670062e80 con 0x7fe6580a3cd0 2024-04-25T20:57:24.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6580a3cd0 msgr2=0x7fe6580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6580a3cd0 0x7fe6580a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe670078fc0 tx=0x7fe664009190 comp rx=0 tx=0).stop 2024-04-25T20:57:24.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006b1f0 msgr2=0x7fe6701019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006b1f0 0x7fe6701019b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe66000d980 tx=0x7fe66000dc90 comp rx=0 tx=0).stop 2024-04-25T20:57:24.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 shutdown_connections 2024-04-25T20:57:24.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe6580a3cd0 0x7fe6580a6190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe67006b1f0 0x7fe6701019b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe670108a60 0x7fe670102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 --2- 172.21.15.47:0/3703188497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67006bbb0 0x7fe670101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 >> 172.21.15.47:0/3703188497 conn(0x7fe6700789e0 msgr2=0x7fe6700ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 shutdown_connections 2024-04-25T20:57:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:24.368+0000 7fe67628e700 1 -- 172.21.15.47:0/3703188497 wait complete. 2024-04-25T20:57:25.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T20:57:25.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:24 smithi186 ceph-mon[32529]: pgmap v308: 97 pgs: 97 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T20:57:25.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:24 smithi186 ceph-mon[32529]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:25.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:24 smithi092 ceph-mon[32524]: pgmap v308: 97 pgs: 97 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T20:57:25.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:24 smithi092 ceph-mon[32524]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:25.316 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:25.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:24 smithi047 ceph-mon[29195]: pgmap v308: 97 pgs: 97 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T20:57:25.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:24 smithi047 ceph-mon[29195]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 -- 172.21.15.47:0/3543122912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268103950 msgr2=0x7f8268103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 --2- 172.21.15.47:0/3543122912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268103950 0x7f8268103d30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f825800b0a0 tx=0x7f825800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 -- 172.21.15.47:0/3543122912 shutdown_connections 2024-04-25T20:57:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 --2- 172.21.15.47:0/3543122912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268103950 0x7f8268103d30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 --2- 172.21.15.47:0/3543122912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8268101ad0 0x7f82681117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 --2- 172.21.15.47:0/3543122912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8268101030 0x7f82681014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 -- 172.21.15.47:0/3543122912 >> 172.21.15.47:0/3543122912 conn(0x7f8268077fb0 msgr2=0x7f82680ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 -- 172.21.15.47:0/3543122912 shutdown_connections 2024-04-25T20:57:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.943+0000 7f826ce14700 1 -- 172.21.15.47:0/3543122912 wait complete. 2024-04-25T20:57:26.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826ce14700 1 Processor -- start 2024-04-25T20:57:26.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826ce14700 1 -- start start 2024-04-25T20:57:26.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826ce14700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268101030 0x7f826819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826ce14700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8268101ad0 0x7f826819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826ce14700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8268103950 0x7f826819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826ce14700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8268114df0 con 0x7f8268101030 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826ce14700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8268114c70 con 0x7f8268103950 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826ce14700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8268114f70 con 0x7f8268101ad0 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268101030 0x7f826819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268101030 0x7f826819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50314/0 (socket says 172.21.15.47:50314) 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f826659c700 1 -- 172.21.15.47:0/2644730041 learned_addr learned my addr 172.21.15.47:0/2644730041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f8266d9d700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8268103950 0x7f826819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.944+0000 7f8265d9b700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8268101ad0 0x7f826819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f826659c700 1 -- 172.21.15.47:0/2644730041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8268101ad0 msgr2=0x7f826819b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f826659c700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8268101ad0 0x7f826819b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f826659c700 1 -- 172.21.15.47:0/2644730041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8268103950 msgr2=0x7f826819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f826659c700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8268103950 0x7f826819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f826659c700 1 -- 172.21.15.47:0/2644730041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82681a0260 con 0x7f8268101030 2024-04-25T20:57:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f826659c700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268101030 0x7f826819b2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f825800b070 tx=0x7f8258004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f82577fe700 1 -- 172.21.15.47:0/2644730041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258021070 con 0x7f8268101030 2024-04-25T20:57:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f82577fe700 1 -- 172.21.15.47:0/2644730041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8258007e30 con 0x7f8268101030 2024-04-25T20:57:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82681a04f0 con 0x7f8268101030 2024-04-25T20:57:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f82577fe700 1 -- 172.21.15.47:0/2644730041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258004780 con 0x7f8268101030 2024-04-25T20:57:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.945+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82681a7dd0 con 0x7f8268101030 2024-04-25T20:57:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.946+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8268102040 con 0x7f8268101030 2024-04-25T20:57:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.947+0000 7f82577fe700 1 -- 172.21.15.47:0/2644730041 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8258007980 con 0x7f8268101030 2024-04-25T20:57:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.948+0000 7f82577fe700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f82500a3d50 0x7f82500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.948+0000 7f8265d9b700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f82500a3d50 0x7f82500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.948+0000 7f82577fe700 1 -- 172.21.15.47:0/2644730041 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f825800f020 con 0x7f8268101030 2024-04-25T20:57:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.948+0000 7f8265d9b700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f82500a3d50 0x7f82500a6210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8268100630 tx=0x7f825c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:26.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:26.952+0000 7f82577fe700 1 -- 172.21.15.47:0/2644730041 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f825802a020 con 0x7f8268101030 2024-04-25T20:57:27.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.194+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8268062e80 con 0x7f82500a3d50 2024-04-25T20:57:27.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.200+0000 7f82577fe700 1 -- 172.21.15.47:0/2644730041 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8268062e80 con 0x7f82500a3d50 2024-04-25T20:57:27.200 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_1/5eababe3-75e2-499f-b960-7d667e1f9f2f 2024-04-25T20:57:27.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f82500a3d50 msgr2=0x7f82500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f82500a3d50 0x7f82500a6210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8268100630 tx=0x7f825c00a380 comp rx=0 tx=0).stop 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268101030 msgr2=0x7f826819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268101030 0x7f826819b2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f825800b070 tx=0x7f8258004d00 comp rx=0 tx=0).stop 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 shutdown_connections 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f82500a3d50 0x7f82500a6210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8268101030 0x7f826819b2a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8268103950 0x7f826819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 --2- 172.21.15.47:0/2644730041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8268101ad0 0x7f826819b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 >> 172.21.15.47:0/2644730041 conn(0x7f8268077fb0 msgr2=0x7f82680fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 shutdown_connections 2024-04-25T20:57:27.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:27.202+0000 7f826ce14700 1 -- 172.21.15.47:0/2644730041 wait complete. 2024-04-25T20:57:27.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:26 smithi047 ceph-mon[29195]: pgmap v309: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-25T20:57:27.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:26 smithi092 ceph-mon[32524]: pgmap v309: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-25T20:57:27.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:26 smithi186 ceph-mon[32529]: pgmap v309: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-25T20:57:27.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:57:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:28.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:27 smithi092 ceph-mon[32524]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:27 smithi047 ceph-mon[29195]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:28.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:27 smithi186 ceph-mon[32529]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:29.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:28 smithi092 ceph-mon[32524]: pgmap v310: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-04-25T20:57:29.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:28 smithi047 ceph-mon[29195]: pgmap v310: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-04-25T20:57:29.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:28 smithi186 ceph-mon[32529]: pgmap v310: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-04-25T20:57:29.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 -- 172.21.15.47:0/2156000660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06bb30 msgr2=0x7fb52c10f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 --2- 172.21.15.47:0/2156000660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06bb30 0x7fb52c10f8d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b0a0 tx=0x7fb52400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 -- 172.21.15.47:0/2156000660 shutdown_connections 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 --2- 172.21.15.47:0/2156000660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06bb30 0x7fb52c10f8d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 --2- 172.21.15.47:0/2156000660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c108a70 0x7fb52c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 --2- 172.21.15.47:0/2156000660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c06b170 0x7fb52c06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 -- 172.21.15.47:0/2156000660 >> 172.21.15.47:0/2156000660 conn(0x7fb52c0777b0 msgr2=0x7fb52c077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 -- 172.21.15.47:0/2156000660 shutdown_connections 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 -- 172.21.15.47:0/2156000660 wait complete. 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.939+0000 7fb531fe7700 1 Processor -- start 2024-04-25T20:57:29.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb531fe7700 1 -- start start 2024-04-25T20:57:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb531fe7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06b170 0x7fb52c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb531fe7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c06bb30 0x7fb52c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb531fe7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c108a70 0x7fb52c19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb531fe7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c112cb0 con 0x7fb52c06b170 2024-04-25T20:57:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb531fe7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c112b30 con 0x7fb52c108a70 2024-04-25T20:57:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb531fe7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c112e30 con 0x7fb52c06bb30 2024-04-25T20:57:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06b170 0x7fb52c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06b170 0x7fb52c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50322/0 (socket says 172.21.15.47:50322) 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb52b7fe700 1 -- 172.21.15.47:0/134791186 learned_addr learned my addr 172.21.15.47:0/134791186 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.940+0000 7fb52bfff700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c108a70 0x7fb52c19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb52affd700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c06bb30 0x7fb52c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb52b7fe700 1 -- 172.21.15.47:0/134791186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c06bb30 msgr2=0x7fb52c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb52b7fe700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c06bb30 0x7fb52c199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb52b7fe700 1 -- 172.21.15.47:0/134791186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c108a70 msgr2=0x7fb52c19d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb52b7fe700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c108a70 0x7fb52c19d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb52b7fe700 1 -- 172.21.15.47:0/134791186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c19e130 con 0x7fb52c06b170 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb52b7fe700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06b170 0x7fb52c199120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00d950 tx=0x7fb51c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb528ff9700 1 -- 172.21.15.47:0/134791186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c016070 con 0x7fb52c06b170 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c19e420 con 0x7fb52c06b170 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb528ff9700 1 -- 172.21.15.47:0/134791186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51c009df0 con 0x7fb52c06b170 2024-04-25T20:57:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb528ff9700 1 -- 172.21.15.47:0/134791186 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00f3c0 con 0x7fb52c06b170 2024-04-25T20:57:29.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.941+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a5ca0 con 0x7fb52c06b170 2024-04-25T20:57:29.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.942+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52c1041a0 con 0x7fb52c06b170 2024-04-25T20:57:29.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.943+0000 7fb528ff9700 1 -- 172.21.15.47:0/134791186 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb51c019070 con 0x7fb52c06b170 2024-04-25T20:57:29.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.944+0000 7fb528ff9700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb5140a3cd0 0x7fb5140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.944+0000 7fb52affd700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb5140a3cd0 0x7fb5140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.944+0000 7fb528ff9700 1 -- 172.21.15.47:0/134791186 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fb51c0cffb0 con 0x7fb52c06b170 2024-04-25T20:57:29.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.944+0000 7fb52affd700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb5140a3cd0 0x7fb5140a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb520006fd0 tx=0x7fb520008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:29.948+0000 7fb528ff9700 1 -- 172.21.15.47:0/134791186 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb51c09c9e0 con 0x7fb52c06b170 2024-04-25T20:57:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.204+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb52c19a480 con 0x7fb52c06b170 2024-04-25T20:57:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.205+0000 7fb528ff9700 1 -- 172.21.15.47:0/134791186 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11853 (secure 0 0 0) 0x7fb51c09c2e0 con 0x7fb52c06b170 2024-04-25T20:57:30.206 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:57:30.206 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:57:11.813557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24488,24467]},"id":1}]} 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb5140a3cd0 msgr2=0x7fb5140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb5140a3cd0 0x7fb5140a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb520006fd0 tx=0x7fb520008040 comp rx=0 tx=0).stop 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06b170 msgr2=0x7fb52c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06b170 0x7fb52c199120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00d950 tx=0x7fb51c00dc60 comp rx=0 tx=0).stop 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 shutdown_connections 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb5140a3cd0 0x7fb5140a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb52c06b170 0x7fb52c199120 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c108a70 0x7fb52c19d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 --2- 172.21.15.47:0/134791186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c06bb30 0x7fb52c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 >> 172.21.15.47:0/134791186 conn(0x7fb52c0777b0 msgr2=0x7fb52c100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 shutdown_connections 2024-04-25T20:57:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:30.207+0000 7fb531fe7700 1 -- 172.21.15.47:0/134791186 wait complete. 2024-04-25T20:57:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2024-04-25T20:57:30.963 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:57:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:31.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:30 smithi092 ceph-mon[32524]: pgmap v311: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-25T20:57:31.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:30 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/134791186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:31.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:30 smithi047 ceph-mon[29195]: pgmap v311: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-25T20:57:31.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:30 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/134791186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:31.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:30 smithi186 ceph-mon[32529]: pgmap v311: 97 pgs: 97 active+clean; 622 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-25T20:57:31.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:30 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/134791186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 -- 172.21.15.47:0/1812324723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c108a30 msgr2=0x7f955c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 --2- 172.21.15.47:0/1812324723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c108a30 0x7f955c108e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b0a0 tx=0x7f954c00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 -- 172.21.15.47:0/1812324723 shutdown_connections 2024-04-25T20:57:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 --2- 172.21.15.47:0/1812324723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c108a30 0x7f955c108e10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 --2- 172.21.15.47:0/1812324723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f955c06b0f0 0x7f955c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 --2- 172.21.15.47:0/1812324723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f955c06a750 0x7f955c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 -- 172.21.15.47:0/1812324723 >> 172.21.15.47:0/1812324723 conn(0x7f955c0788f0 msgr2=0x7f955c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 -- 172.21.15.47:0/1812324723 shutdown_connections 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.852+0000 7f956399b700 1 -- 172.21.15.47:0/1812324723 wait complete. 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f956399b700 1 Processor -- start 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f956399b700 1 -- start start 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f956399b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c06a750 0x7f955c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f956399b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f955c06b0f0 0x7f955c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f956399b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f955c108a30 0x7f955c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f956399b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f955c112bb0 con 0x7f955c06a750 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f956399b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f955c112a30 con 0x7f955c108a30 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f956399b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f955c112d30 con 0x7f955c06b0f0 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f9561737700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c06a750 0x7f955c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:32.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f9561737700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c06a750 0x7f955c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50334/0 (socket says 172.21.15.47:50334) 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.853+0000 7f9561737700 1 -- 172.21.15.47:0/2375214012 learned_addr learned my addr 172.21.15.47:0/2375214012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f9561737700 1 -- 172.21.15.47:0/2375214012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f955c06b0f0 msgr2=0x7f955c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f9561737700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f955c06b0f0 0x7f955c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f9561f38700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f955c108a30 0x7f955c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f9561737700 1 -- 172.21.15.47:0/2375214012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f955c108a30 msgr2=0x7f955c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f9561737700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f955c108a30 0x7f955c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f9561737700 1 -- 172.21.15.47:0/2375214012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c1a0390 con 0x7f955c06a750 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f9561f38700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f955c108a30 0x7f955c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f9561737700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c06a750 0x7f955c19b380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b070 tx=0x7f954c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f955a7fc700 1 -- 172.21.15.47:0/2375214012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c021070 con 0x7f955c06a750 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f955a7fc700 1 -- 172.21.15.47:0/2375214012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f954c003c00 con 0x7f955c06a750 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c1a0620 con 0x7f955c06a750 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f955a7fc700 1 -- 172.21.15.47:0/2375214012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c004880 con 0x7f955c06a750 2024-04-25T20:57:32.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.854+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c1a7f00 con 0x7f955c06a750 2024-04-25T20:57:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.855+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c069730 con 0x7f955c06a750 2024-04-25T20:57:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.856+0000 7f955a7fc700 1 -- 172.21.15.47:0/2375214012 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f954c004020 con 0x7f955c06a750 2024-04-25T20:57:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.857+0000 7f955a7fc700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f95440a3cd0 0x7f95440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.857+0000 7f9560f36700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f95440a3cd0 0x7f95440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.857+0000 7f955a7fc700 1 -- 172.21.15.47:0/2375214012 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f954c00f020 con 0x7f955c06a750 2024-04-25T20:57:32.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.857+0000 7f9560f36700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f95440a3cd0 0x7f95440a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f955c079010 tx=0x7f9550005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:32.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:32.861+0000 7f955a7fc700 1 -- 172.21.15.47:0/2375214012 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f954c02a020 con 0x7f955c06a750 2024-04-25T20:57:33.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.071+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f955c02df80 con 0x7f955c06a750 2024-04-25T20:57:33.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:32 smithi047 ceph-mon[29195]: pgmap v312: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-04-25T20:57:33.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.073+0000 7f955a7fc700 1 -- 172.21.15.47:0/2375214012 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22160 (secure 0 0 0) 0x7f954c0a4020 con 0x7f955c06a750 2024-04-25T20:57:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:57:33.074 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:56:53.086017+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:57:33.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f95440a3cd0 msgr2=0x7f95440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:33.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f95440a3cd0 0x7f95440a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f955c079010 tx=0x7f9550005c80 comp rx=0 tx=0).stop 2024-04-25T20:57:33.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c06a750 msgr2=0x7f955c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:33.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c06a750 0x7f955c19b380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b070 tx=0x7f954c004db0 comp rx=0 tx=0).stop 2024-04-25T20:57:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 shutdown_connections 2024-04-25T20:57:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f95440a3cd0 0x7f95440a6190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955c06a750 0x7f955c19b380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f955c108a30 0x7f955c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 --2- 172.21.15.47:0/2375214012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f955c06b0f0 0x7f955c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 >> 172.21.15.47:0/2375214012 conn(0x7f955c0788f0 msgr2=0x7f955c10df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 shutdown_connections 2024-04-25T20:57:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:33.075+0000 7f956399b700 1 -- 172.21.15.47:0/2375214012 wait complete. 2024-04-25T20:57:33.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:32 smithi092 ceph-mon[32524]: pgmap v312: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-04-25T20:57:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:32 smithi186 ceph-mon[32529]: pgmap v312: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-04-25T20:57:33.728 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T20:57:33.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:57:33.983 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:34.282 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:33 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2375214012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:34.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:33 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2375214012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:34.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:33 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2375214012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:35.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:34 smithi092 ceph-mon[32524]: pgmap v313: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-25T20:57:35.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:34 smithi047 ceph-mon[29195]: pgmap v313: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-25T20:57:35.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:34 smithi186 ceph-mon[32529]: pgmap v313: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-25T20:57:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 -- 172.21.15.47:0/2147843381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f68108a60 msgr2=0x7f3f68108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/2147843381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f68108a60 0x7f3f68108e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009990 tx=0x7f3f64009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 -- 172.21.15.47:0/2147843381 shutdown_connections 2024-04-25T20:57:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/2147843381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f68108a60 0x7f3f68108e40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/2147843381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f6806bbb0 0x7f3f6810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/2147843381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f6806b1f0 0x7f3f6806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 -- 172.21.15.47:0/2147843381 >> 172.21.15.47:0/2147843381 conn(0x7f3f680789e0 msgr2=0x7f3f68078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 -- 172.21.15.47:0/2147843381 shutdown_connections 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.580+0000 7f3f6fc71700 1 -- 172.21.15.47:0/2147843381 wait complete. 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6fc71700 1 Processor -- start 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6fc71700 1 -- start start 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6fc71700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f6806b1f0 0x7f3f681019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6fc71700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f6806bbb0 0x7f3f68101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6fc71700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f68108a60 0x7f3f68102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6fc71700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68112ca0 con 0x7f3f6806b1f0 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6fc71700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68112b20 con 0x7f3f6806bbb0 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6fc71700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68112e20 con 0x7f3f68108a60 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6da0d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f6806b1f0 0x7f3f681019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6da0d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f6806b1f0 0x7f3f681019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32826/0 (socket says 172.21.15.47:32826) 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6da0d700 1 -- 172.21.15.47:0/840401793 learned_addr learned my addr 172.21.15.47:0/840401793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6d20c700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f6806bbb0 0x7f3f68101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6e20e700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f68108a60 0x7f3f68102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6da0d700 1 -- 172.21.15.47:0/840401793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f68108a60 msgr2=0x7f3f68102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6da0d700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f68108a60 0x7f3f68102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6da0d700 1 -- 172.21.15.47:0/840401793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f6806bbb0 msgr2=0x7f3f68101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6da0d700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f6806bbb0 0x7f3f68101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6da0d700 1 -- 172.21.15.47:0/840401793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f681a7710 con 0x7f3f6806b1f0 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6d20c700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f6806bbb0 0x7f3f68101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.581+0000 7f3f6e20e700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f68108a60 0x7f3f68102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.582+0000 7f3f6da0d700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f6806b1f0 0x7f3f681019b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3f64000c00 tx=0x7f3f640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.582+0000 7f3f5effd700 1 -- 172.21.15.47:0/840401793 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64027070 con 0x7f3f6806b1f0 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.582+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f681a79a0 con 0x7f3f6806b1f0 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.582+0000 7f3f5effd700 1 -- 172.21.15.47:0/840401793 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f64003fa0 con 0x7f3f6806b1f0 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.582+0000 7f3f5effd700 1 -- 172.21.15.47:0/840401793 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64022db0 con 0x7f3f6806b1f0 2024-04-25T20:57:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.582+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f681a7e50 con 0x7f3f6806b1f0 2024-04-25T20:57:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.583+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6810b060 con 0x7f3f6806b1f0 2024-04-25T20:57:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.584+0000 7f3f5effd700 1 -- 172.21.15.47:0/840401793 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3f6402a070 con 0x7f3f6806b1f0 2024-04-25T20:57:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.585+0000 7f3f5effd700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f540a3d50 0x7f3f540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.585+0000 7f3f6d20c700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f540a3d50 0x7f3f540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.585+0000 7f3f5effd700 1 -- 172.21.15.47:0/840401793 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f3f640d1020 con 0x7f3f6806b1f0 2024-04-25T20:57:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.585+0000 7f3f6d20c700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f540a3d50 0x7f3f540a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f68079050 tx=0x7f3f58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:35.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.589+0000 7f3f5effd700 1 -- 172.21.15.47:0/840401793 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f64013070 con 0x7f3f6806b1f0 2024-04-25T20:57:35.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.794+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f68062e80 con 0x7f3f540a3d50 2024-04-25T20:57:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.802+0000 7f3f5effd700 1 -- 172.21.15.47:0/840401793 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258689 (secure 0 0 0) 0x7f3f68062e80 con 0x7f3f540a3d50 2024-04-25T20:57:35.806 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:57:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.804+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f540a3d50 msgr2=0x7f3f540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.804+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f540a3d50 0x7f3f540a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f68079050 tx=0x7f3f58008040 comp rx=0 tx=0).stop 2024-04-25T20:57:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.804+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f6806b1f0 msgr2=0x7f3f681019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.804+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f6806b1f0 0x7f3f681019b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3f64000c00 tx=0x7f3f640047b0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.805+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 shutdown_connections 2024-04-25T20:57:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.805+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f540a3d50 0x7f3f540a6210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.805+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f6806b1f0 0x7f3f681019b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.805+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f68108a60 0x7f3f68102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.805+0000 7f3f6fc71700 1 --2- 172.21.15.47:0/840401793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f6806bbb0 0x7f3f68101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.805+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 >> 172.21.15.47:0/840401793 conn(0x7f3f680789e0 msgr2=0x7f3f680ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.805+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 shutdown_connections 2024-04-25T20:57:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:35.805+0000 7f3f6fc71700 1 -- 172.21.15.47:0/840401793 wait complete. 2024-04-25T20:57:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-25T20:57:36.474 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-04-25T20:57:33.909308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649946,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":341,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362592,"kb_used_data":13440,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579232,"statfs":{"total":1151940427776,"available":1151569133568,"internally_reserved":0,"allocated":13762560,"data_stored":7547358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32739,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":73,"num_read_kb":63,"num_write":20,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003158"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.292228+0000","last_change":"2024-04-25T20:56:51.283722+0000","last_active":"2024-04-25T20:56:53.292228+0000","last_peered":"2024-04-25T20:56:53.292228+0000","last_clean":"2024-04-25T20:56:53.292228+0000","last_became_active":"2024-04-25T20:56:51.283359+0000","last_became_peered":"2024-04-25T20:56:51.283359+0000","last_unstale":"2024-04-25T20:56:53.292228+0000","last_undegraded":"2024-04-25T20:56:53.292228+0000","last_fullsized":"2024-04-25T20:56:53.292228+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:47.314687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288263+0000","last_change":"2024-04-25T20:56:51.273199+0000","last_active":"2024-04-25T20:56:53.288263+0000","last_peered":"2024-04-25T20:56:53.288263+0000","last_clean":"2024-04-25T20:56:53.288263+0000","last_became_active":"2024-04-25T20:56:51.272838+0000","last_became_peered":"2024-04-25T20:56:51.272838+0000","last_unstale":"2024-04-25T20:56:53.288263+0000","last_undegraded":"2024-04-25T20:56:53.288263+0000","last_fullsized":"2024-04-25T20:56:53.288263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:58.033315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288053+0000","last_change":"2024-04-25T20:56:51.281461+0000","last_active":"2024-04-25T20:56:53.288053+0000","last_peered":"2024-04-25T20:56:53.288053+0000","last_clean":"2024-04-25T20:56:53.288053+0000","last_became_active":"2024-04-25T20:56:51.281141+0000","last_became_peered":"2024-04-25T20:56:51.281141+0000","last_unstale":"2024-04-25T20:56:53.288053+0000","last_undegraded":"2024-04-25T20:56:53.288053+0000","last_fullsized":"2024-04-25T20:56:53.288053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286802+0000","last_change":"2024-04-25T20:56:51.283187+0000","last_active":"2024-04-25T20:56:53.286802+0000","last_peered":"2024-04-25T20:56:53.286802+0000","last_clean":"2024-04-25T20:56:53.286802+0000","last_became_active":"2024-04-25T20:56:51.282640+0000","last_became_peered":"2024-04-25T20:56:51.282640+0000","last_unstale":"2024-04-25T20:56:53.286802+0000","last_undegraded":"2024-04-25T20:56:53.286802+0000","last_fullsized":"2024-04-25T20:56:53.286802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:15.588694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285423+0000","last_change":"2024-04-25T20:56:51.273204+0000","last_active":"2024-04-25T20:56:53.285423+0000","last_peered":"2024-04-25T20:56:53.285423+0000","last_clean":"2024-04-25T20:56:53.285423+0000","last_became_active":"2024-04-25T20:56:51.272126+0000","last_became_peered":"2024-04-25T20:56:51.272126+0000","last_unstale":"2024-04-25T20:56:53.285423+0000","last_undegraded":"2024-04-25T20:56:53.285423+0000","last_fullsized":"2024-04-25T20:56:53.285423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:31.684294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284080+0000","last_change":"2024-04-25T20:56:51.270762+0000","last_active":"2024-04-25T20:56:53.284080+0000","last_peered":"2024-04-25T20:56:53.284080+0000","last_clean":"2024-04-25T20:56:53.284080+0000","last_became_active":"2024-04-25T20:56:51.270444+0000","last_became_peered":"2024-04-25T20:56:51.270444+0000","last_unstale":"2024-04-25T20:56:53.284080+0000","last_undegraded":"2024-04-25T20:56:53.284080+0000","last_fullsized":"2024-04-25T20:56:53.284080+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:00.155417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273637+0000","last_change":"2024-04-25T20:56:51.276029+0000","last_active":"2024-04-25T20:56:53.273637+0000","last_peered":"2024-04-25T20:56:53.273637+0000","last_clean":"2024-04-25T20:56:53.273637+0000","last_became_active":"2024-04-25T20:56:51.275272+0000","last_became_peered":"2024-04-25T20:56:51.275272+0000","last_unstale":"2024-04-25T20:56:53.273637+0000","last_undegraded":"2024-04-25T20:56:53.273637+0000","last_fullsized":"2024-04-25T20:56:53.273637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.428444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284911+0000","last_change":"2024-04-25T20:56:51.273769+0000","last_active":"2024-04-25T20:56:53.284911+0000","last_peered":"2024-04-25T20:56:53.284911+0000","last_clean":"2024-04-25T20:56:53.284911+0000","last_became_active":"2024-04-25T20:56:51.273412+0000","last_became_peered":"2024-04-25T20:56:51.273412+0000","last_unstale":"2024-04-25T20:56:53.284911+0000","last_undegraded":"2024-04-25T20:56:53.284911+0000","last_fullsized":"2024-04-25T20:56:53.284911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.324332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288814+0000","last_change":"2024-04-25T20:56:51.277097+0000","last_active":"2024-04-25T20:56:53.288814+0000","last_peered":"2024-04-25T20:56:53.288814+0000","last_clean":"2024-04-25T20:56:53.288814+0000","last_became_active":"2024-04-25T20:56:51.276768+0000","last_became_peered":"2024-04-25T20:56:51.276768+0000","last_unstale":"2024-04-25T20:56:53.288814+0000","last_undegraded":"2024-04-25T20:56:53.288814+0000","last_fullsized":"2024-04-25T20:56:53.288814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:23.276470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285214+0000","last_change":"2024-04-25T20:56:51.279046+0000","last_active":"2024-04-25T20:56:53.285214+0000","last_peered":"2024-04-25T20:56:53.285214+0000","last_clean":"2024-04-25T20:56:53.285214+0000","last_became_active":"2024-04-25T20:56:51.278701+0000","last_became_peered":"2024-04-25T20:56:51.278701+0000","last_unstale":"2024-04-25T20:56:53.285214+0000","last_undegraded":"2024-04-25T20:56:53.285214+0000","last_fullsized":"2024-04-25T20:56:53.285214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.370482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274307+0000","last_change":"2024-04-25T20:56:51.271593+0000","last_active":"2024-04-25T20:56:53.274307+0000","last_peered":"2024-04-25T20:56:53.274307+0000","last_clean":"2024-04-25T20:56:53.274307+0000","last_became_active":"2024-04-25T20:56:51.271224+0000","last_became_peered":"2024-04-25T20:56:51.271224+0000","last_unstale":"2024-04-25T20:56:53.274307+0000","last_undegraded":"2024-04-25T20:56:53.274307+0000","last_fullsized":"2024-04-25T20:56:53.274307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:46.847929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286421+0000","last_change":"2024-04-25T20:56:51.278281+0000","last_active":"2024-04-25T20:56:53.286421+0000","last_peered":"2024-04-25T20:56:53.286421+0000","last_clean":"2024-04-25T20:56:53.286421+0000","last_became_active":"2024-04-25T20:56:51.277947+0000","last_became_peered":"2024-04-25T20:56:51.277947+0000","last_unstale":"2024-04-25T20:56:53.286421+0000","last_undegraded":"2024-04-25T20:56:53.286421+0000","last_fullsized":"2024-04-25T20:56:53.286421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:02.509296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284164+0000","last_change":"2024-04-25T20:56:51.273784+0000","last_active":"2024-04-25T20:56:53.284164+0000","last_peered":"2024-04-25T20:56:53.284164+0000","last_clean":"2024-04-25T20:56:53.284164+0000","last_became_active":"2024-04-25T20:56:51.273489+0000","last_became_peered":"2024-04-25T20:56:51.273489+0000","last_unstale":"2024-04-25T20:56:53.284164+0000","last_undegraded":"2024-04-25T20:56:53.284164+0000","last_fullsized":"2024-04-25T20:56:53.284164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:00.765186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287961+0000","last_change":"2024-04-25T20:56:51.269829+0000","last_active":"2024-04-25T20:56:53.287961+0000","last_peered":"2024-04-25T20:56:53.287961+0000","last_clean":"2024-04-25T20:56:53.287961+0000","last_became_active":"2024-04-25T20:56:51.269446+0000","last_became_peered":"2024-04-25T20:56:51.269446+0000","last_unstale":"2024-04-25T20:56:53.287961+0000","last_undegraded":"2024-04-25T20:56:53.287961+0000","last_fullsized":"2024-04-25T20:56:53.287961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:12.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287557+0000","last_change":"2024-04-25T20:56:51.278117+0000","last_active":"2024-04-25T20:56:53.287557+0000","last_peered":"2024-04-25T20:56:53.287557+0000","last_clean":"2024-04-25T20:56:53.287557+0000","last_became_active":"2024-04-25T20:56:51.277596+0000","last_became_peered":"2024-04-25T20:56:51.277596+0000","last_unstale":"2024-04-25T20:56:53.287557+0000","last_undegraded":"2024-04-25T20:56:53.287557+0000","last_fullsized":"2024-04-25T20:56:53.287557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:28.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287275+0000","last_change":"2024-04-25T20:56:51.281575+0000","last_active":"2024-04-25T20:56:53.287275+0000","last_peered":"2024-04-25T20:56:53.287275+0000","last_clean":"2024-04-25T20:56:53.287275+0000","last_became_active":"2024-04-25T20:56:51.281217+0000","last_became_peered":"2024-04-25T20:56:51.281217+0000","last_unstale":"2024-04-25T20:56:53.287275+0000","last_undegraded":"2024-04-25T20:56:53.287275+0000","last_fullsized":"2024-04-25T20:56:53.287275+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:05.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.292384+0000","last_change":"2024-04-25T20:56:51.284608+0000","last_active":"2024-04-25T20:56:53.292384+0000","last_peered":"2024-04-25T20:56:53.292384+0000","last_clean":"2024-04-25T20:56:53.292384+0000","last_became_active":"2024-04-25T20:56:51.284257+0000","last_became_peered":"2024-04-25T20:56:51.284257+0000","last_unstale":"2024-04-25T20:56:53.292384+0000","last_undegraded":"2024-04-25T20:56:53.292384+0000","last_fullsized":"2024-04-25T20:56:53.292384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:44:59.184192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.272282+0000","last_change":"2024-04-25T20:56:51.276770+0000","last_active":"2024-04-25T20:56:53.272282+0000","last_peered":"2024-04-25T20:56:53.272282+0000","last_clean":"2024-04-25T20:56:53.272282+0000","last_became_active":"2024-04-25T20:56:51.276431+0000","last_became_peered":"2024-04-25T20:56:51.276431+0000","last_unstale":"2024-04-25T20:56:53.272282+0000","last_undegraded":"2024-04-25T20:56:53.272282+0000","last_fullsized":"2024-04-25T20:56:53.272282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:14:25.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273277+0000","last_change":"2024-04-25T20:56:51.283728+0000","last_active":"2024-04-25T20:56:53.273277+0000","last_peered":"2024-04-25T20:56:53.273277+0000","last_clean":"2024-04-25T20:56:53.273277+0000","last_became_active":"2024-04-25T20:56:51.283370+0000","last_became_peered":"2024-04-25T20:56:51.283370+0000","last_unstale":"2024-04-25T20:56:53.273277+0000","last_undegraded":"2024-04-25T20:56:53.273277+0000","last_fullsized":"2024-04-25T20:56:53.273277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:18.042769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.725258+0000","last_change":"2024-04-25T20:56:48.259737+0000","last_active":"2024-04-25T20:57:06.725258+0000","last_peered":"2024-04-25T20:57:06.725258+0000","last_clean":"2024-04-25T20:57:06.725258+0000","last_became_active":"2024-04-25T20:56:48.259442+0000","last_became_peered":"2024-04-25T20:56:48.259442+0000","last_unstale":"2024-04-25T20:57:06.725258+0000","last_undegraded":"2024-04-25T20:57:06.725258+0000","last_fullsized":"2024-04-25T20:57:06.725258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:23.191536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285403+0000","last_change":"2024-04-25T20:56:51.274510+0000","last_active":"2024-04-25T20:56:53.285403+0000","last_peered":"2024-04-25T20:56:53.285403+0000","last_clean":"2024-04-25T20:56:53.285403+0000","last_became_active":"2024-04-25T20:56:51.274137+0000","last_became_peered":"2024-04-25T20:56:51.274137+0000","last_unstale":"2024-04-25T20:56:53.285403+0000","last_undegraded":"2024-04-25T20:56:53.285403+0000","last_fullsized":"2024-04-25T20:56:53.285403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:05.903617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.292596+0000","last_change":"2024-04-25T20:56:48.947850+0000","last_active":"2024-04-25T20:56:53.292596+0000","last_peered":"2024-04-25T20:56:53.292596+0000","last_clean":"2024-04-25T20:56:53.292596+0000","last_became_active":"2024-04-25T20:56:48.946160+0000","last_became_peered":"2024-04-25T20:56:48.946160+0000","last_unstale":"2024-04-25T20:56:53.292596+0000","last_undegraded":"2024-04-25T20:56:53.292596+0000","last_fullsized":"2024-04-25T20:56:53.292596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:32.692390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288569+0000","last_change":"2024-04-25T20:56:51.280478+0000","last_active":"2024-04-25T20:56:53.288569+0000","last_peered":"2024-04-25T20:56:53.288569+0000","last_clean":"2024-04-25T20:56:53.288569+0000","last_became_active":"2024-04-25T20:56:51.280173+0000","last_became_peered":"2024-04-25T20:56:51.280173+0000","last_unstale":"2024-04-25T20:56:53.288569+0000","last_undegraded":"2024-04-25T20:56:53.288569+0000","last_fullsized":"2024-04-25T20:56:53.288569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:48.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.722657+0000","last_change":"2024-04-25T20:56:48.263494+0000","last_active":"2024-04-25T20:57:06.722657+0000","last_peered":"2024-04-25T20:57:06.722657+0000","last_clean":"2024-04-25T20:57:06.722657+0000","last_became_active":"2024-04-25T20:56:48.262726+0000","last_became_peered":"2024-04-25T20:56:48.262726+0000","last_unstale":"2024-04-25T20:57:06.722657+0000","last_undegraded":"2024-04-25T20:57:06.722657+0000","last_fullsized":"2024-04-25T20:57:06.722657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:07.616369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287499+0000","last_change":"2024-04-25T20:56:51.278383+0000","last_active":"2024-04-25T20:56:53.287499+0000","last_peered":"2024-04-25T20:56:53.287499+0000","last_clean":"2024-04-25T20:56:53.287499+0000","last_became_active":"2024-04-25T20:56:51.277603+0000","last_became_peered":"2024-04-25T20:56:51.277603+0000","last_unstale":"2024-04-25T20:56:53.287499+0000","last_undegraded":"2024-04-25T20:56:53.287499+0000","last_fullsized":"2024-04-25T20:56:53.287499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:12.197940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291784+0000","last_change":"2024-04-25T20:56:48.258869+0000","last_active":"2024-04-25T20:56:53.291784+0000","last_peered":"2024-04-25T20:56:53.291784+0000","last_clean":"2024-04-25T20:56:53.291784+0000","last_became_active":"2024-04-25T20:56:48.258414+0000","last_became_peered":"2024-04-25T20:56:48.258414+0000","last_unstale":"2024-04-25T20:56:53.291784+0000","last_undegraded":"2024-04-25T20:56:53.291784+0000","last_fullsized":"2024-04-25T20:56:53.291784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:09.889936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286416+0000","last_change":"2024-04-25T20:56:51.276579+0000","last_active":"2024-04-25T20:56:53.286416+0000","last_peered":"2024-04-25T20:56:53.286416+0000","last_clean":"2024-04-25T20:56:53.286416+0000","last_became_active":"2024-04-25T20:56:51.276252+0000","last_became_peered":"2024-04-25T20:56:51.276252+0000","last_unstale":"2024-04-25T20:56:53.286416+0000","last_undegraded":"2024-04-25T20:56:53.286416+0000","last_fullsized":"2024-04-25T20:56:53.286416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:06.764359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:31.770328+0000","last_change":"2024-04-25T20:56:48.262543+0000","last_active":"2024-04-25T20:57:31.770328+0000","last_peered":"2024-04-25T20:57:31.770328+0000","last_clean":"2024-04-25T20:57:31.770328+0000","last_became_active":"2024-04-25T20:56:48.262086+0000","last_became_peered":"2024-04-25T20:56:48.262086+0000","last_unstale":"2024-04-25T20:57:31.770328+0000","last_undegraded":"2024-04-25T20:57:31.770328+0000","last_fullsized":"2024-04-25T20:57:31.770328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:43.722131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284819+0000","last_change":"2024-04-25T20:56:51.273500+0000","last_active":"2024-04-25T20:56:53.284819+0000","last_peered":"2024-04-25T20:56:53.284819+0000","last_clean":"2024-04-25T20:56:53.284819+0000","last_became_active":"2024-04-25T20:56:51.272882+0000","last_became_peered":"2024-04-25T20:56:51.272882+0000","last_unstale":"2024-04-25T20:56:53.284819+0000","last_undegraded":"2024-04-25T20:56:53.284819+0000","last_fullsized":"2024-04-25T20:56:53.284819+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:05.682258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.723704+0000","last_change":"2024-04-25T20:56:48.255225+0000","last_active":"2024-04-25T20:57:06.723704+0000","last_peered":"2024-04-25T20:57:06.723704+0000","last_clean":"2024-04-25T20:57:06.723704+0000","last_became_active":"2024-04-25T20:56:48.254644+0000","last_became_peered":"2024-04-25T20:56:48.254644+0000","last_unstale":"2024-04-25T20:57:06.723704+0000","last_undegraded":"2024-04-25T20:57:06.723704+0000","last_fullsized":"2024-04-25T20:57:06.723704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:35.724612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286680+0000","last_change":"2024-04-25T20:56:51.270709+0000","last_active":"2024-04-25T20:56:53.286680+0000","last_peered":"2024-04-25T20:56:53.286680+0000","last_clean":"2024-04-25T20:56:53.286680+0000","last_became_active":"2024-04-25T20:56:51.270295+0000","last_became_peered":"2024-04-25T20:56:51.270295+0000","last_unstale":"2024-04-25T20:56:53.286680+0000","last_undegraded":"2024-04-25T20:56:53.286680+0000","last_fullsized":"2024-04-25T20:56:53.286680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:41.998953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.723299+0000","last_change":"2024-04-25T20:56:48.256498+0000","last_active":"2024-04-25T20:57:06.723299+0000","last_peered":"2024-04-25T20:57:06.723299+0000","last_clean":"2024-04-25T20:57:06.723299+0000","last_became_active":"2024-04-25T20:56:48.256140+0000","last_became_peered":"2024-04-25T20:56:48.256140+0000","last_unstale":"2024-04-25T20:57:06.723299+0000","last_undegraded":"2024-04-25T20:57:06.723299+0000","last_fullsized":"2024-04-25T20:57:06.723299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:41.458235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284998+0000","last_change":"2024-04-25T20:56:51.267575+0000","last_active":"2024-04-25T20:56:53.284998+0000","last_peered":"2024-04-25T20:56:53.284998+0000","last_clean":"2024-04-25T20:56:53.284998+0000","last_became_active":"2024-04-25T20:56:51.267265+0000","last_became_peered":"2024-04-25T20:56:51.267265+0000","last_unstale":"2024-04-25T20:56:53.284998+0000","last_undegraded":"2024-04-25T20:56:53.284998+0000","last_fullsized":"2024-04-25T20:56:53.284998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:30.760555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285718+0000","last_change":"2024-04-25T20:56:48.260747+0000","last_active":"2024-04-25T20:56:53.285718+0000","last_peered":"2024-04-25T20:56:53.285718+0000","last_clean":"2024-04-25T20:56:53.285718+0000","last_became_active":"2024-04-25T20:56:48.260339+0000","last_became_peered":"2024-04-25T20:56:48.260339+0000","last_unstale":"2024-04-25T20:56:53.285718+0000","last_undegraded":"2024-04-25T20:56:53.285718+0000","last_fullsized":"2024-04-25T20:56:53.285718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:19.769631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.271980+0000","last_change":"2024-04-25T20:56:51.268365+0000","last_active":"2024-04-25T20:56:53.271980+0000","last_peered":"2024-04-25T20:56:53.271980+0000","last_clean":"2024-04-25T20:56:53.271980+0000","last_became_active":"2024-04-25T20:56:51.268048+0000","last_became_peered":"2024-04-25T20:56:51.268048+0000","last_unstale":"2024-04-25T20:56:53.271980+0000","last_undegraded":"2024-04-25T20:56:53.271980+0000","last_fullsized":"2024-04-25T20:56:53.271980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:31.763147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:28.765113+0000","last_change":"2024-04-25T20:56:48.256113+0000","last_active":"2024-04-25T20:57:28.765113+0000","last_peered":"2024-04-25T20:57:28.765113+0000","last_clean":"2024-04-25T20:57:28.765113+0000","last_became_active":"2024-04-25T20:56:48.255644+0000","last_became_peered":"2024-04-25T20:56:48.255644+0000","last_unstale":"2024-04-25T20:57:28.765113+0000","last_undegraded":"2024-04-25T20:57:28.765113+0000","last_fullsized":"2024-04-25T20:57:28.765113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:12.100099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288156+0000","last_change":"2024-04-25T20:56:51.276990+0000","last_active":"2024-04-25T20:56:53.288156+0000","last_peered":"2024-04-25T20:56:53.288156+0000","last_clean":"2024-04-25T20:56:53.288156+0000","last_became_active":"2024-04-25T20:56:51.276638+0000","last_became_peered":"2024-04-25T20:56:51.276638+0000","last_unstale":"2024-04-25T20:56:53.288156+0000","last_undegraded":"2024-04-25T20:56:53.288156+0000","last_fullsized":"2024-04-25T20:56:53.288156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:06.645278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.713242+0000","last_change":"2024-04-25T20:56:48.263216+0000","last_active":"2024-04-25T20:57:05.713242+0000","last_peered":"2024-04-25T20:57:05.713242+0000","last_clean":"2024-04-25T20:57:05.713242+0000","last_became_active":"2024-04-25T20:56:48.262854+0000","last_became_peered":"2024-04-25T20:56:48.262854+0000","last_unstale":"2024-04-25T20:57:05.713242+0000","last_undegraded":"2024-04-25T20:57:05.713242+0000","last_fullsized":"2024-04-25T20:57:05.713242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:00.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288446+0000","last_change":"2024-04-25T20:56:51.278479+0000","last_active":"2024-04-25T20:56:53.288446+0000","last_peered":"2024-04-25T20:56:53.288446+0000","last_clean":"2024-04-25T20:56:53.288446+0000","last_became_active":"2024-04-25T20:56:51.276285+0000","last_became_peered":"2024-04-25T20:56:51.276285+0000","last_unstale":"2024-04-25T20:56:53.288446+0000","last_undegraded":"2024-04-25T20:56:53.288446+0000","last_fullsized":"2024-04-25T20:56:53.288446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:14.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.712562+0000","last_change":"2024-04-25T20:56:48.946808+0000","last_active":"2024-04-25T20:57:05.712562+0000","last_peered":"2024-04-25T20:57:05.712562+0000","last_clean":"2024-04-25T20:57:05.712562+0000","last_became_active":"2024-04-25T20:56:48.946421+0000","last_became_peered":"2024-04-25T20:56:48.946421+0000","last_unstale":"2024-04-25T20:57:05.712562+0000","last_undegraded":"2024-04-25T20:57:05.712562+0000","last_fullsized":"2024-04-25T20:57:05.712562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:47.763950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288512+0000","last_change":"2024-04-25T20:56:51.274761+0000","last_active":"2024-04-25T20:56:53.288512+0000","last_peered":"2024-04-25T20:56:53.288512+0000","last_clean":"2024-04-25T20:56:53.288512+0000","last_became_active":"2024-04-25T20:56:51.274452+0000","last_became_peered":"2024-04-25T20:56:51.274452+0000","last_unstale":"2024-04-25T20:56:53.288512+0000","last_undegraded":"2024-04-25T20:56:53.288512+0000","last_fullsized":"2024-04-25T20:56:53.288512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:32.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'25","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:29.767575+0000","last_change":"2024-04-25T20:56:48.258982+0000","last_active":"2024-04-25T20:57:29.767575+0000","last_peered":"2024-04-25T20:57:29.767575+0000","last_clean":"2024-04-25T20:57:29.767575+0000","last_became_active":"2024-04-25T20:56:48.258593+0000","last_became_peered":"2024-04-25T20:56:48.258593+0000","last_unstale":"2024-04-25T20:57:29.767575+0000","last_undegraded":"2024-04-25T20:57:29.767575+0000","last_fullsized":"2024-04-25T20:57:29.767575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:20.518202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54952,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":78,"num_write":26,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285754+0000","last_change":"2024-04-25T20:56:51.276361+0000","last_active":"2024-04-25T20:56:53.285754+0000","last_peered":"2024-04-25T20:56:53.285754+0000","last_clean":"2024-04-25T20:56:53.285754+0000","last_became_active":"2024-04-25T20:56:51.276045+0000","last_became_peered":"2024-04-25T20:56:51.276045+0000","last_unstale":"2024-04-25T20:56:53.285754+0000","last_undegraded":"2024-04-25T20:56:53.285754+0000","last_fullsized":"2024-04-25T20:56:53.285754+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:04.325068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.723933+0000","last_change":"2024-04-25T20:56:48.252358+0000","last_active":"2024-04-25T20:57:06.723933+0000","last_peered":"2024-04-25T20:57:06.723933+0000","last_clean":"2024-04-25T20:57:06.723933+0000","last_became_active":"2024-04-25T20:56:48.252027+0000","last_became_peered":"2024-04-25T20:56:48.252027+0000","last_unstale":"2024-04-25T20:57:06.723933+0000","last_undegraded":"2024-04-25T20:57:06.723933+0000","last_fullsized":"2024-04-25T20:57:06.723933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:03.024487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.283842+0000","last_change":"2024-04-25T20:56:51.279599+0000","last_active":"2024-04-25T20:56:53.283842+0000","last_peered":"2024-04-25T20:56:53.283842+0000","last_clean":"2024-04-25T20:56:53.283842+0000","last_became_active":"2024-04-25T20:56:51.279294+0000","last_became_peered":"2024-04-25T20:56:51.279294+0000","last_unstale":"2024-04-25T20:56:53.283842+0000","last_undegraded":"2024-04-25T20:56:53.283842+0000","last_fullsized":"2024-04-25T20:56:53.283842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.712468+0000","last_change":"2024-04-25T20:56:48.261005+0000","last_active":"2024-04-25T20:57:05.712468+0000","last_peered":"2024-04-25T20:57:05.712468+0000","last_clean":"2024-04-25T20:57:05.712468+0000","last_became_active":"2024-04-25T20:56:48.260543+0000","last_became_peered":"2024-04-25T20:56:48.260543+0000","last_unstale":"2024-04-25T20:57:05.712468+0000","last_undegraded":"2024-04-25T20:57:05.712468+0000","last_fullsized":"2024-04-25T20:57:05.712468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:41:54.326036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274121+0000","last_change":"2024-04-25T20:56:51.271753+0000","last_active":"2024-04-25T20:56:53.274121+0000","last_peered":"2024-04-25T20:56:53.274121+0000","last_clean":"2024-04-25T20:56:53.274121+0000","last_became_active":"2024-04-25T20:56:51.271392+0000","last_became_peered":"2024-04-25T20:56:51.271392+0000","last_unstale":"2024-04-25T20:56:53.274121+0000","last_undegraded":"2024-04-25T20:56:53.274121+0000","last_fullsized":"2024-04-25T20:56:53.274121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:26.605647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:31.770158+0000","last_change":"2024-04-25T20:56:48.260729+0000","last_active":"2024-04-25T20:57:31.770158+0000","last_peered":"2024-04-25T20:57:31.770158+0000","last_clean":"2024-04-25T20:57:31.770158+0000","last_became_active":"2024-04-25T20:56:48.260377+0000","last_became_peered":"2024-04-25T20:56:48.260377+0000","last_unstale":"2024-04-25T20:57:31.770158+0000","last_undegraded":"2024-04-25T20:57:31.770158+0000","last_fullsized":"2024-04-25T20:57:31.770158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:21.582266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273509+0000","last_change":"2024-04-25T20:56:51.275691+0000","last_active":"2024-04-25T20:56:53.273509+0000","last_peered":"2024-04-25T20:56:53.273509+0000","last_clean":"2024-04-25T20:56:53.273509+0000","last_became_active":"2024-04-25T20:56:51.275220+0000","last_became_peered":"2024-04-25T20:56:51.275220+0000","last_unstale":"2024-04-25T20:56:53.273509+0000","last_undegraded":"2024-04-25T20:56:53.273509+0000","last_fullsized":"2024-04-25T20:56:53.273509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:24.367413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.724849+0000","last_change":"2024-04-25T20:56:48.946465+0000","last_active":"2024-04-25T20:57:06.724849+0000","last_peered":"2024-04-25T20:57:06.724849+0000","last_clean":"2024-04-25T20:57:06.724849+0000","last_became_active":"2024-04-25T20:56:48.946093+0000","last_became_peered":"2024-04-25T20:56:48.946093+0000","last_unstale":"2024-04-25T20:57:06.724849+0000","last_undegraded":"2024-04-25T20:57:06.724849+0000","last_fullsized":"2024-04-25T20:57:06.724849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:29.502073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286688+0000","last_change":"2024-04-25T20:56:51.278464+0000","last_active":"2024-04-25T20:56:53.286688+0000","last_peered":"2024-04-25T20:56:53.286688+0000","last_clean":"2024-04-25T20:56:53.286688+0000","last_became_active":"2024-04-25T20:56:51.278119+0000","last_became_peered":"2024-04-25T20:56:51.278119+0000","last_unstale":"2024-04-25T20:56:53.286688+0000","last_undegraded":"2024-04-25T20:56:53.286688+0000","last_fullsized":"2024-04-25T20:56:53.286688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:57:37.224817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:07.718866+0000","last_change":"2024-04-25T20:56:48.255123+0000","last_active":"2024-04-25T20:57:07.718866+0000","last_peered":"2024-04-25T20:57:07.718866+0000","last_clean":"2024-04-25T20:57:07.718866+0000","last_became_active":"2024-04-25T20:56:48.254680+0000","last_became_peered":"2024-04-25T20:56:48.254680+0000","last_unstale":"2024-04-25T20:57:07.718866+0000","last_undegraded":"2024-04-25T20:57:07.718866+0000","last_fullsized":"2024-04-25T20:57:07.718866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:24.089590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291463+0000","last_change":"2024-04-25T20:56:51.283640+0000","last_active":"2024-04-25T20:56:53.291463+0000","last_peered":"2024-04-25T20:56:53.291463+0000","last_clean":"2024-04-25T20:56:53.291463+0000","last_became_active":"2024-04-25T20:56:51.283307+0000","last_became_peered":"2024-04-25T20:56:51.283307+0000","last_unstale":"2024-04-25T20:56:53.291463+0000","last_undegraded":"2024-04-25T20:56:53.291463+0000","last_fullsized":"2024-04-25T20:56:53.291463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:30.087816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.272015+0000","last_change":"2024-04-25T20:56:48.947508+0000","last_active":"2024-04-25T20:56:53.272015+0000","last_peered":"2024-04-25T20:56:53.272015+0000","last_clean":"2024-04-25T20:56:53.272015+0000","last_became_active":"2024-04-25T20:56:48.946582+0000","last_became_peered":"2024-04-25T20:56:48.946582+0000","last_unstale":"2024-04-25T20:56:53.272015+0000","last_undegraded":"2024-04-25T20:56:53.272015+0000","last_fullsized":"2024-04-25T20:56:53.272015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:11.198667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284330+0000","last_change":"2024-04-25T20:56:51.273560+0000","last_active":"2024-04-25T20:56:53.284330+0000","last_peered":"2024-04-25T20:56:53.284330+0000","last_clean":"2024-04-25T20:56:53.284330+0000","last_became_active":"2024-04-25T20:56:51.273182+0000","last_became_peered":"2024-04-25T20:56:51.273182+0000","last_unstale":"2024-04-25T20:56:53.284330+0000","last_undegraded":"2024-04-25T20:56:53.284330+0000","last_fullsized":"2024-04-25T20:56:53.284330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:47.012771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273833+0000","last_change":"2024-04-25T20:56:48.260754+0000","last_active":"2024-04-25T20:56:53.273833+0000","last_peered":"2024-04-25T20:56:53.273833+0000","last_clean":"2024-04-25T20:56:53.273833+0000","last_became_active":"2024-04-25T20:56:48.260420+0000","last_became_peered":"2024-04-25T20:56:48.260420+0000","last_unstale":"2024-04-25T20:56:53.273833+0000","last_undegraded":"2024-04-25T20:56:53.273833+0000","last_fullsized":"2024-04-25T20:56:53.273833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:00.219023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.283925+0000","last_change":"2024-04-25T20:56:51.271944+0000","last_active":"2024-04-25T20:56:53.283925+0000","last_peered":"2024-04-25T20:56:53.283925+0000","last_clean":"2024-04-25T20:56:53.283925+0000","last_became_active":"2024-04-25T20:56:51.270656+0000","last_became_peered":"2024-04-25T20:56:51.270656+0000","last_unstale":"2024-04-25T20:56:53.283925+0000","last_undegraded":"2024-04-25T20:56:53.283925+0000","last_fullsized":"2024-04-25T20:56:53.283925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:00.624888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286455+0000","last_change":"2024-04-25T20:56:51.281207+0000","last_active":"2024-04-25T20:56:53.286455+0000","last_peered":"2024-04-25T20:56:53.286455+0000","last_clean":"2024-04-25T20:56:53.286455+0000","last_became_active":"2024-04-25T20:56:51.280879+0000","last_became_peered":"2024-04-25T20:56:51.280879+0000","last_unstale":"2024-04-25T20:56:53.286455+0000","last_undegraded":"2024-04-25T20:56:53.286455+0000","last_fullsized":"2024-04-25T20:56:53.286455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:01:48.356766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:29.766244+0000","last_change":"2024-04-25T20:56:48.946471+0000","last_active":"2024-04-25T20:57:29.766244+0000","last_peered":"2024-04-25T20:57:29.766244+0000","last_clean":"2024-04-25T20:57:29.766244+0000","last_became_active":"2024-04-25T20:56:48.946029+0000","last_became_peered":"2024-04-25T20:56:48.946029+0000","last_unstale":"2024-04-25T20:57:29.766244+0000","last_undegraded":"2024-04-25T20:57:29.766244+0000","last_fullsized":"2024-04-25T20:57:29.766244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:30:16.400157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285884+0000","last_change":"2024-04-25T20:56:51.278521+0000","last_active":"2024-04-25T20:56:53.285884+0000","last_peered":"2024-04-25T20:56:53.285884+0000","last_clean":"2024-04-25T20:56:53.285884+0000","last_became_active":"2024-04-25T20:56:51.278224+0000","last_became_peered":"2024-04-25T20:56:51.278224+0000","last_unstale":"2024-04-25T20:56:53.285884+0000","last_undegraded":"2024-04-25T20:56:53.285884+0000","last_fullsized":"2024-04-25T20:56:53.285884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:57.211010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:08.731857+0000","last_change":"2024-04-25T20:56:48.258479+0000","last_active":"2024-04-25T20:57:08.731857+0000","last_peered":"2024-04-25T20:57:08.731857+0000","last_clean":"2024-04-25T20:57:08.731857+0000","last_became_active":"2024-04-25T20:56:48.257951+0000","last_became_peered":"2024-04-25T20:56:48.257951+0000","last_unstale":"2024-04-25T20:57:08.731857+0000","last_undegraded":"2024-04-25T20:57:08.731857+0000","last_fullsized":"2024-04-25T20:57:08.731857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:13.812201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273477+0000","last_change":"2024-04-25T20:55:28.701357+0000","last_active":"2024-04-25T20:56:53.273477+0000","last_peered":"2024-04-25T20:56:53.273477+0000","last_clean":"2024-04-25T20:56:53.273477+0000","last_became_active":"2024-04-25T20:55:28.700906+0000","last_became_peered":"2024-04-25T20:55:28.700906+0000","last_unstale":"2024-04-25T20:56:53.273477+0000","last_undegraded":"2024-04-25T20:56:53.273477+0000","last_fullsized":"2024-04-25T20:56:53.273477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_clean_scrub_stamp":"2024-04-25T20:52:10.334323+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:37:35.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.283530+0000","last_change":"2024-04-25T20:56:51.275249+0000","last_active":"2024-04-25T20:56:53.283530+0000","last_peered":"2024-04-25T20:56:53.283530+0000","last_clean":"2024-04-25T20:56:53.283530+0000","last_became_active":"2024-04-25T20:56:51.274938+0000","last_became_peered":"2024-04-25T20:56:51.274938+0000","last_unstale":"2024-04-25T20:56:53.283530+0000","last_undegraded":"2024-04-25T20:56:53.283530+0000","last_fullsized":"2024-04-25T20:56:53.283530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:53.933324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.723812+0000","last_change":"2024-04-25T20:56:48.259357+0000","last_active":"2024-04-25T20:57:06.723812+0000","last_peered":"2024-04-25T20:57:06.723812+0000","last_clean":"2024-04-25T20:57:06.723812+0000","last_became_active":"2024-04-25T20:56:48.258840+0000","last_became_peered":"2024-04-25T20:56:48.258840+0000","last_unstale":"2024-04-25T20:57:06.723812+0000","last_undegraded":"2024-04-25T20:57:06.723812+0000","last_fullsized":"2024-04-25T20:57:06.723812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:30.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.712232+0000","last_change":"2024-04-25T20:56:48.946498+0000","last_active":"2024-04-25T20:57:05.712232+0000","last_peered":"2024-04-25T20:57:05.712232+0000","last_clean":"2024-04-25T20:57:05.712232+0000","last_became_active":"2024-04-25T20:56:48.946124+0000","last_became_peered":"2024-04-25T20:56:48.946124+0000","last_unstale":"2024-04-25T20:57:05.712232+0000","last_undegraded":"2024-04-25T20:57:05.712232+0000","last_fullsized":"2024-04-25T20:57:05.712232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:08.545574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284086+0000","last_change":"2024-04-25T20:56:51.276093+0000","last_active":"2024-04-25T20:56:53.284086+0000","last_peered":"2024-04-25T20:56:53.284086+0000","last_clean":"2024-04-25T20:56:53.284086+0000","last_became_active":"2024-04-25T20:56:51.275771+0000","last_became_peered":"2024-04-25T20:56:51.275771+0000","last_unstale":"2024-04-25T20:56:53.284086+0000","last_undegraded":"2024-04-25T20:56:53.284086+0000","last_fullsized":"2024-04-25T20:56:53.284086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:15.159208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:27.287236+0000","last_change":"2024-04-25T20:56:48.251519+0000","last_active":"2024-04-25T20:57:27.287236+0000","last_peered":"2024-04-25T20:57:27.287236+0000","last_clean":"2024-04-25T20:57:27.287236+0000","last_became_active":"2024-04-25T20:56:48.251071+0000","last_became_peered":"2024-04-25T20:56:48.251071+0000","last_unstale":"2024-04-25T20:57:27.287236+0000","last_undegraded":"2024-04-25T20:57:27.287236+0000","last_fullsized":"2024-04-25T20:57:27.287236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:24.583520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285514+0000","last_change":"2024-04-25T20:56:51.275461+0000","last_active":"2024-04-25T20:56:53.285514+0000","last_peered":"2024-04-25T20:56:53.285514+0000","last_clean":"2024-04-25T20:56:53.285514+0000","last_became_active":"2024-04-25T20:56:51.275087+0000","last_became_peered":"2024-04-25T20:56:51.275087+0000","last_unstale":"2024-04-25T20:56:53.285514+0000","last_undegraded":"2024-04-25T20:56:53.285514+0000","last_fullsized":"2024-04-25T20:56:53.285514+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:01:02.514490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.711481+0000","last_change":"2024-04-25T20:56:49.031177+0000","last_active":"2024-04-25T20:57:05.711481+0000","last_peered":"2024-04-25T20:57:05.711481+0000","last_clean":"2024-04-25T20:57:05.711481+0000","last_became_active":"2024-04-25T20:56:49.030697+0000","last_became_peered":"2024-04-25T20:56:49.030697+0000","last_unstale":"2024-04-25T20:57:05.711481+0000","last_undegraded":"2024-04-25T20:57:05.711481+0000","last_fullsized":"2024-04-25T20:57:05.711481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:13.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288295+0000","last_change":"2024-04-25T20:56:51.271157+0000","last_active":"2024-04-25T20:56:53.288295+0000","last_peered":"2024-04-25T20:56:53.288295+0000","last_clean":"2024-04-25T20:56:53.288295+0000","last_became_active":"2024-04-25T20:56:51.270795+0000","last_became_peered":"2024-04-25T20:56:51.270795+0000","last_unstale":"2024-04-25T20:56:53.288295+0000","last_undegraded":"2024-04-25T20:56:53.288295+0000","last_fullsized":"2024-04-25T20:56:53.288295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:36.741663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.713214+0000","last_change":"2024-04-25T20:56:48.946689+0000","last_active":"2024-04-25T20:57:05.713214+0000","last_peered":"2024-04-25T20:57:05.713214+0000","last_clean":"2024-04-25T20:57:05.713214+0000","last_became_active":"2024-04-25T20:56:48.946229+0000","last_became_peered":"2024-04-25T20:56:48.946229+0000","last_unstale":"2024-04-25T20:57:05.713214+0000","last_undegraded":"2024-04-25T20:57:05.713214+0000","last_fullsized":"2024-04-25T20:57:05.713214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:04.120197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284722+0000","last_change":"2024-04-25T20:56:51.275657+0000","last_active":"2024-04-25T20:56:53.284722+0000","last_peered":"2024-04-25T20:56:53.284722+0000","last_clean":"2024-04-25T20:56:53.284722+0000","last_became_active":"2024-04-25T20:56:51.275237+0000","last_became_peered":"2024-04-25T20:56:51.275237+0000","last_unstale":"2024-04-25T20:56:53.284722+0000","last_undegraded":"2024-04-25T20:56:53.284722+0000","last_fullsized":"2024-04-25T20:56:53.284722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:39.778340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.712632+0000","last_change":"2024-04-25T20:56:48.251882+0000","last_active":"2024-04-25T20:57:05.712632+0000","last_peered":"2024-04-25T20:57:05.712632+0000","last_clean":"2024-04-25T20:57:05.712632+0000","last_became_active":"2024-04-25T20:56:48.251571+0000","last_became_peered":"2024-04-25T20:56:48.251571+0000","last_unstale":"2024-04-25T20:57:05.712632+0000","last_undegraded":"2024-04-25T20:57:05.712632+0000","last_fullsized":"2024-04-25T20:57:05.712632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:51.769332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291952+0000","last_change":"2024-04-25T20:56:51.280648+0000","last_active":"2024-04-25T20:56:53.291952+0000","last_peered":"2024-04-25T20:56:53.291952+0000","last_clean":"2024-04-25T20:56:53.291952+0000","last_became_active":"2024-04-25T20:56:51.280232+0000","last_became_peered":"2024-04-25T20:56:51.280232+0000","last_unstale":"2024-04-25T20:56:53.291952+0000","last_undegraded":"2024-04-25T20:56:53.291952+0000","last_fullsized":"2024-04-25T20:56:53.291952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:12.375519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273407+0000","last_change":"2024-04-25T20:56:48.251976+0000","last_active":"2024-04-25T20:56:53.273407+0000","last_peered":"2024-04-25T20:56:53.273407+0000","last_clean":"2024-04-25T20:56:53.273407+0000","last_became_active":"2024-04-25T20:56:48.251549+0000","last_became_peered":"2024-04-25T20:56:48.251549+0000","last_unstale":"2024-04-25T20:56:53.273407+0000","last_undegraded":"2024-04-25T20:56:53.273407+0000","last_fullsized":"2024-04-25T20:56:53.273407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:27.413462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285162+0000","last_change":"2024-04-25T20:56:51.277166+0000","last_active":"2024-04-25T20:56:53.285162+0000","last_peered":"2024-04-25T20:56:53.285162+0000","last_clean":"2024-04-25T20:56:53.285162+0000","last_became_active":"2024-04-25T20:56:51.276840+0000","last_became_peered":"2024-04-25T20:56:51.276840+0000","last_unstale":"2024-04-25T20:56:53.285162+0000","last_undegraded":"2024-04-25T20:56:53.285162+0000","last_fullsized":"2024-04-25T20:56:53.285162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:52.756660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.713394+0000","last_change":"2024-04-25T20:56:48.263426+0000","last_active":"2024-04-25T20:57:05.713394+0000","last_peered":"2024-04-25T20:57:05.713394+0000","last_clean":"2024-04-25T20:57:05.713394+0000","last_became_active":"2024-04-25T20:56:48.262832+0000","last_became_peered":"2024-04-25T20:56:48.262832+0000","last_unstale":"2024-04-25T20:57:05.713394+0000","last_undegraded":"2024-04-25T20:57:05.713394+0000","last_fullsized":"2024-04-25T20:57:05.713394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:09.362161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274265+0000","last_change":"2024-04-25T20:56:51.274267+0000","last_active":"2024-04-25T20:56:53.274265+0000","last_peered":"2024-04-25T20:56:53.274265+0000","last_clean":"2024-04-25T20:56:53.274265+0000","last_became_active":"2024-04-25T20:56:51.273807+0000","last_became_peered":"2024-04-25T20:56:51.273807+0000","last_unstale":"2024-04-25T20:56:53.274265+0000","last_undegraded":"2024-04-25T20:56:53.274265+0000","last_fullsized":"2024-04-25T20:56:53.274265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.724101+0000","last_change":"2024-04-25T20:56:49.031011+0000","last_active":"2024-04-25T20:57:06.724101+0000","last_peered":"2024-04-25T20:57:06.724101+0000","last_clean":"2024-04-25T20:57:06.724101+0000","last_became_active":"2024-04-25T20:56:49.030522+0000","last_became_peered":"2024-04-25T20:56:49.030522+0000","last_unstale":"2024-04-25T20:57:06.724101+0000","last_undegraded":"2024-04-25T20:57:06.724101+0000","last_fullsized":"2024-04-25T20:57:06.724101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:00.915454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274301+0000","last_change":"2024-04-25T20:56:51.276989+0000","last_active":"2024-04-25T20:56:53.274301+0000","last_peered":"2024-04-25T20:56:53.274301+0000","last_clean":"2024-04-25T20:56:53.274301+0000","last_became_active":"2024-04-25T20:56:51.275417+0000","last_became_peered":"2024-04-25T20:56:51.275417+0000","last_unstale":"2024-04-25T20:56:53.274301+0000","last_undegraded":"2024-04-25T20:56:53.274301+0000","last_fullsized":"2024-04-25T20:56:53.274301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:27.516062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.290382+0000","last_change":"2024-04-25T20:56:48.253713+0000","last_active":"2024-04-25T20:56:53.290382+0000","last_peered":"2024-04-25T20:56:53.290382+0000","last_clean":"2024-04-25T20:56:53.290382+0000","last_became_active":"2024-04-25T20:56:48.253215+0000","last_became_peered":"2024-04-25T20:56:48.253215+0000","last_unstale":"2024-04-25T20:56:53.290382+0000","last_undegraded":"2024-04-25T20:56:53.290382+0000","last_fullsized":"2024-04-25T20:56:53.290382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:57.946840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:54.481839+0000","last_change":"2024-04-25T20:56:51.274691+0000","last_active":"2024-04-25T20:56:54.481839+0000","last_peered":"2024-04-25T20:56:54.481839+0000","last_clean":"2024-04-25T20:56:54.481839+0000","last_became_active":"2024-04-25T20:56:51.274350+0000","last_became_peered":"2024-04-25T20:56:51.274350+0000","last_unstale":"2024-04-25T20:56:54.481839+0000","last_undegraded":"2024-04-25T20:56:54.481839+0000","last_fullsized":"2024-04-25T20:56:54.481839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:52.375409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286977+0000","last_change":"2024-04-25T20:56:48.251616+0000","last_active":"2024-04-25T20:56:53.286977+0000","last_peered":"2024-04-25T20:56:53.286977+0000","last_clean":"2024-04-25T20:56:53.286977+0000","last_became_active":"2024-04-25T20:56:48.251248+0000","last_became_peered":"2024-04-25T20:56:48.251248+0000","last_unstale":"2024-04-25T20:56:53.286977+0000","last_undegraded":"2024-04-25T20:56:53.286977+0000","last_fullsized":"2024-04-25T20:56:53.286977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:51.032356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274201+0000","last_change":"2024-04-25T20:56:51.278372+0000","last_active":"2024-04-25T20:56:53.274201+0000","last_peered":"2024-04-25T20:56:53.274201+0000","last_clean":"2024-04-25T20:56:53.274201+0000","last_became_active":"2024-04-25T20:56:51.277989+0000","last_became_peered":"2024-04-25T20:56:51.277989+0000","last_unstale":"2024-04-25T20:56:53.274201+0000","last_undegraded":"2024-04-25T20:56:53.274201+0000","last_fullsized":"2024-04-25T20:56:53.274201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.017713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.292018+0000","last_change":"2024-04-25T20:56:51.276270+0000","last_active":"2024-04-25T20:56:53.292018+0000","last_peered":"2024-04-25T20:56:53.292018+0000","last_clean":"2024-04-25T20:56:53.292018+0000","last_became_active":"2024-04-25T20:56:51.275948+0000","last_became_peered":"2024-04-25T20:56:51.275948+0000","last_unstale":"2024-04-25T20:56:53.292018+0000","last_undegraded":"2024-04-25T20:56:53.292018+0000","last_fullsized":"2024-04-25T20:56:53.292018+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:33.548439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.272969+0000","last_change":"2024-04-25T20:56:51.283931+0000","last_active":"2024-04-25T20:56:53.272969+0000","last_peered":"2024-04-25T20:56:53.272969+0000","last_clean":"2024-04-25T20:56:53.272969+0000","last_became_active":"2024-04-25T20:56:51.283120+0000","last_became_peered":"2024-04-25T20:56:51.283120+0000","last_unstale":"2024-04-25T20:56:53.272969+0000","last_undegraded":"2024-04-25T20:56:53.272969+0000","last_fullsized":"2024-04-25T20:56:53.272969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:26.676750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.272910+0000","last_change":"2024-04-25T20:56:51.286013+0000","last_active":"2024-04-25T20:56:53.272910+0000","last_peered":"2024-04-25T20:56:53.272910+0000","last_clean":"2024-04-25T20:56:53.272910+0000","last_became_active":"2024-04-25T20:56:51.285645+0000","last_became_peered":"2024-04-25T20:56:51.285645+0000","last_unstale":"2024-04-25T20:56:53.272910+0000","last_undegraded":"2024-04-25T20:56:53.272910+0000","last_fullsized":"2024-04-25T20:56:53.272910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:51.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287881+0000","last_change":"2024-04-25T20:56:51.272567+0000","last_active":"2024-04-25T20:56:53.287881+0000","last_peered":"2024-04-25T20:56:53.287881+0000","last_clean":"2024-04-25T20:56:53.287881+0000","last_became_active":"2024-04-25T20:56:51.271427+0000","last_became_peered":"2024-04-25T20:56:51.271427+0000","last_unstale":"2024-04-25T20:56:53.287881+0000","last_undegraded":"2024-04-25T20:56:53.287881+0000","last_fullsized":"2024-04-25T20:56:53.287881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:19.945982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284492+0000","last_change":"2024-04-25T20:56:51.272448+0000","last_active":"2024-04-25T20:56:53.284492+0000","last_peered":"2024-04-25T20:56:53.284492+0000","last_clean":"2024-04-25T20:56:53.284492+0000","last_became_active":"2024-04-25T20:56:51.271965+0000","last_became_peered":"2024-04-25T20:56:51.271965+0000","last_unstale":"2024-04-25T20:56:53.284492+0000","last_undegraded":"2024-04-25T20:56:53.284492+0000","last_fullsized":"2024-04-25T20:56:53.284492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:03.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291752+0000","last_change":"2024-04-25T20:56:51.280260+0000","last_active":"2024-04-25T20:56:53.291752+0000","last_peered":"2024-04-25T20:56:53.291752+0000","last_clean":"2024-04-25T20:56:53.291752+0000","last_became_active":"2024-04-25T20:56:51.279864+0000","last_became_peered":"2024-04-25T20:56:51.279864+0000","last_unstale":"2024-04-25T20:56:53.291752+0000","last_undegraded":"2024-04-25T20:56:53.291752+0000","last_fullsized":"2024-04-25T20:56:53.291752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.136824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273458+0000","last_change":"2024-04-25T20:56:51.279041+0000","last_active":"2024-04-25T20:56:53.273458+0000","last_peered":"2024-04-25T20:56:53.273458+0000","last_clean":"2024-04-25T20:56:53.273458+0000","last_became_active":"2024-04-25T20:56:51.278302+0000","last_became_peered":"2024-04-25T20:56:51.278302+0000","last_unstale":"2024-04-25T20:56:53.273458+0000","last_undegraded":"2024-04-25T20:56:53.273458+0000","last_fullsized":"2024-04-25T20:56:53.273458+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:02.510919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285280+0000","last_change":"2024-04-25T20:56:51.275618+0000","last_active":"2024-04-25T20:56:53.285280+0000","last_peered":"2024-04-25T20:56:53.285280+0000","last_clean":"2024-04-25T20:56:53.285280+0000","last_became_active":"2024-04-25T20:56:51.275249+0000","last_became_peered":"2024-04-25T20:56:51.275249+0000","last_unstale":"2024-04-25T20:56:53.285280+0000","last_undegraded":"2024-04-25T20:56:53.285280+0000","last_fullsized":"2024-04-25T20:56:53.285280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:01.793083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285925+0000","last_change":"2024-04-25T20:56:51.275868+0000","last_active":"2024-04-25T20:56:53.285925+0000","last_peered":"2024-04-25T20:56:53.285925+0000","last_clean":"2024-04-25T20:56:53.285925+0000","last_became_active":"2024-04-25T20:56:51.275544+0000","last_became_peered":"2024-04-25T20:56:51.275544+0000","last_unstale":"2024-04-25T20:56:53.285925+0000","last_undegraded":"2024-04-25T20:56:53.285925+0000","last_fullsized":"2024-04-25T20:56:53.285925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:31.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286522+0000","last_change":"2024-04-25T20:56:51.274963+0000","last_active":"2024-04-25T20:56:53.286522+0000","last_peered":"2024-04-25T20:56:53.286522+0000","last_clean":"2024-04-25T20:56:53.286522+0000","last_became_active":"2024-04-25T20:56:51.274597+0000","last_became_peered":"2024-04-25T20:56:51.274597+0000","last_unstale":"2024-04-25T20:56:53.286522+0000","last_undegraded":"2024-04-25T20:56:53.286522+0000","last_fullsized":"2024-04-25T20:56:53.286522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:14.464560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288511+0000","last_change":"2024-04-25T20:56:51.279602+0000","last_active":"2024-04-25T20:56:53.288511+0000","last_peered":"2024-04-25T20:56:53.288511+0000","last_clean":"2024-04-25T20:56:53.288511+0000","last_became_active":"2024-04-25T20:56:51.279221+0000","last_became_peered":"2024-04-25T20:56:51.279221+0000","last_unstale":"2024-04-25T20:56:53.288511+0000","last_undegraded":"2024-04-25T20:56:53.288511+0000","last_fullsized":"2024-04-25T20:56:53.288511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:26.960492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.283988+0000","last_change":"2024-04-25T20:56:51.280465+0000","last_active":"2024-04-25T20:56:53.283988+0000","last_peered":"2024-04-25T20:56:53.283988+0000","last_clean":"2024-04-25T20:56:53.283988+0000","last_became_active":"2024-04-25T20:56:51.280168+0000","last_became_peered":"2024-04-25T20:56:51.280168+0000","last_unstale":"2024-04-25T20:56:53.283988+0000","last_undegraded":"2024-04-25T20:56:53.283988+0000","last_fullsized":"2024-04-25T20:56:53.283988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.211113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291065+0000","last_change":"2024-04-25T20:56:51.283084+0000","last_active":"2024-04-25T20:56:53.291065+0000","last_peered":"2024-04-25T20:56:53.291065+0000","last_clean":"2024-04-25T20:56:53.291065+0000","last_became_active":"2024-04-25T20:56:51.282692+0000","last_became_peered":"2024-04-25T20:56:51.282692+0000","last_unstale":"2024-04-25T20:56:53.291065+0000","last_undegraded":"2024-04-25T20:56:53.291065+0000","last_fullsized":"2024-04-25T20:56:53.291065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:13:27.527112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59344,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":166,"num_read_kb":143,"num_write":158,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":178032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":467042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1658880,"data_stored":1113866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":467278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037798,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":467198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1059519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462960,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40444,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704708,"statfs":{"total":95995035648,"available":95953620992,"internally_reserved":0,"allocated":970752,"data_stored":466667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":521412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":467938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":523153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":466978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":467495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724598,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1058812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:57:36.478 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:57:36.729 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:37.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:36 smithi047 ceph-mon[29195]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:37.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:36 smithi047 ceph-mon[29195]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-25T20:57:37.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:36 smithi092 ceph-mon[32524]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:37.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:36 smithi092 ceph-mon[32524]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-25T20:57:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:36 smithi186 ceph-mon[32529]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:36 smithi186 ceph-mon[32529]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/2651891705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 msgr2=0x7f4d68108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/2651891705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 0x7f4d68108e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/2651891705 shutdown_connections 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/2651891705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 0x7f4d68108e40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/2651891705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6806bbb0 0x7f4d6810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/2651891705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6806b1f0 0x7f4d6806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/2651891705 >> 172.21.15.47:0/2651891705 conn(0x7f4d680789e0 msgr2=0x7f4d68078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/2651891705 shutdown_connections 2024-04-25T20:57:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.400+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/2651891705 wait complete. 2024-04-25T20:57:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6d0d4700 1 Processor -- start 2024-04-25T20:57:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6d0d4700 1 -- start start 2024-04-25T20:57:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6d0d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6806b1f0 0x7f4d681019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6d0d4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6806bbb0 0x7f4d68101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6d0d4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 0x7f4d68102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6d0d4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68112ca0 con 0x7f4d68108a60 2024-04-25T20:57:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6d0d4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68112b20 con 0x7f4d6806b1f0 2024-04-25T20:57:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6d0d4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68112e20 con 0x7f4d6806bbb0 2024-04-25T20:57:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 0x7f4d68102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 0x7f4d68102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32858/0 (socket says 172.21.15.47:32858) 2024-04-25T20:57:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.401+0000 7f4d6759e700 1 -- 172.21.15.47:0/3589841833 learned_addr learned my addr 172.21.15.47:0/3589841833 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d66d9d700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6806b1f0 0x7f4d681019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6659c700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6806bbb0 0x7f4d68101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6759e700 1 -- 172.21.15.47:0/3589841833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6806bbb0 msgr2=0x7f4d68101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6759e700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6806bbb0 0x7f4d68101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6759e700 1 -- 172.21.15.47:0/3589841833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6806b1f0 msgr2=0x7f4d681019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6759e700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6806b1f0 0x7f4d681019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6759e700 1 -- 172.21.15.47:0/3589841833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d681a76b0 con 0x7f4d68108a60 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6659c700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6806bbb0 0x7f4d68101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6759e700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 0x7f4d68102430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000b970 tx=0x7f4d6000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d4ffff700 1 -- 172.21.15.47:0/3589841833 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60019070 con 0x7f4d68108a60 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d681a79a0 con 0x7f4d68108a60 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d4ffff700 1 -- 172.21.15.47:0/3589841833 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d60014440 con 0x7f4d68108a60 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d4ffff700 1 -- 172.21.15.47:0/3589841833 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000ce60 con 0x7f4d68108a60 2024-04-25T20:57:38.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.402+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d681a7e90 con 0x7f4d68108a60 2024-04-25T20:57:38.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.403+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6810b060 con 0x7f4d68108a60 2024-04-25T20:57:38.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.404+0000 7f4d4ffff700 1 -- 172.21.15.47:0/3589841833 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4d6001c070 con 0x7f4d68108a60 2024-04-25T20:57:38.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.405+0000 7f4d4ffff700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d500a3cd0 0x7f4d500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:38.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.405+0000 7f4d66d9d700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d500a3cd0 0x7f4d500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:38.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.405+0000 7f4d4ffff700 1 -- 172.21.15.47:0/3589841833 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f4d600cf720 con 0x7f4d68108a60 2024-04-25T20:57:38.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.405+0000 7f4d66d9d700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d500a3cd0 0x7f4d500a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006010 tx=0x7f4d5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:38.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.410+0000 7f4d4ffff700 1 -- 172.21.15.47:0/3589841833 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d6009c260 con 0x7f4d68108a60 2024-04-25T20:57:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.613+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d68102e90 con 0x7f4d500a3cd0 2024-04-25T20:57:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.621+0000 7f4d4ffff700 1 -- 172.21.15.47:0/3589841833 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258690 (secure 0 0 0) 0x7f4d68102e90 con 0x7f4d500a3cd0 2024-04-25T20:57:38.626 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d500a3cd0 msgr2=0x7f4d500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d500a3cd0 0x7f4d500a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006010 tx=0x7f4d5801a040 comp rx=0 tx=0).stop 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 msgr2=0x7f4d68102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 0x7f4d68102430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000b970 tx=0x7f4d6000bd30 comp rx=0 tx=0).stop 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 shutdown_connections 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4d500a3cd0 0x7f4d500a6190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d68108a60 0x7f4d68102430 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6806b1f0 0x7f4d681019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 --2- 172.21.15.47:0/3589841833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d6806bbb0 0x7f4d68101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 >> 172.21.15.47:0/3589841833 conn(0x7f4d680789e0 msgr2=0x7f4d680ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 shutdown_connections 2024-04-25T20:57:38.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:38.625+0000 7f4d6d0d4700 1 -- 172.21.15.47:0/3589841833 wait complete. 2024-04-25T20:57:38.629 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-25T20:57:39.191 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:38 smithi047 ceph-mon[29195]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T20:57:39.191 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:38 smithi047 ceph-mon[29195]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:39.220 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-04-25T20:57:37.910474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651084,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":323,"num_read_kb":279,"num_write":347,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362592,"kb_used_data":13440,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579232,"statfs":{"total":1151940427776,"available":1151569133568,"internally_reserved":0,"allocated":13762560,"data_stored":7550772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13889,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":50,"num_write":15,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003177"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.292228+0000","last_change":"2024-04-25T20:56:51.283722+0000","last_active":"2024-04-25T20:56:53.292228+0000","last_peered":"2024-04-25T20:56:53.292228+0000","last_clean":"2024-04-25T20:56:53.292228+0000","last_became_active":"2024-04-25T20:56:51.283359+0000","last_became_peered":"2024-04-25T20:56:51.283359+0000","last_unstale":"2024-04-25T20:56:53.292228+0000","last_undegraded":"2024-04-25T20:56:53.292228+0000","last_fullsized":"2024-04-25T20:56:53.292228+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:47.314687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288263+0000","last_change":"2024-04-25T20:56:51.273199+0000","last_active":"2024-04-25T20:56:53.288263+0000","last_peered":"2024-04-25T20:56:53.288263+0000","last_clean":"2024-04-25T20:56:53.288263+0000","last_became_active":"2024-04-25T20:56:51.272838+0000","last_became_peered":"2024-04-25T20:56:51.272838+0000","last_unstale":"2024-04-25T20:56:53.288263+0000","last_undegraded":"2024-04-25T20:56:53.288263+0000","last_fullsized":"2024-04-25T20:56:53.288263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:58.033315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288053+0000","last_change":"2024-04-25T20:56:51.281461+0000","last_active":"2024-04-25T20:56:53.288053+0000","last_peered":"2024-04-25T20:56:53.288053+0000","last_clean":"2024-04-25T20:56:53.288053+0000","last_became_active":"2024-04-25T20:56:51.281141+0000","last_became_peered":"2024-04-25T20:56:51.281141+0000","last_unstale":"2024-04-25T20:56:53.288053+0000","last_undegraded":"2024-04-25T20:56:53.288053+0000","last_fullsized":"2024-04-25T20:56:53.288053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286802+0000","last_change":"2024-04-25T20:56:51.283187+0000","last_active":"2024-04-25T20:56:53.286802+0000","last_peered":"2024-04-25T20:56:53.286802+0000","last_clean":"2024-04-25T20:56:53.286802+0000","last_became_active":"2024-04-25T20:56:51.282640+0000","last_became_peered":"2024-04-25T20:56:51.282640+0000","last_unstale":"2024-04-25T20:56:53.286802+0000","last_undegraded":"2024-04-25T20:56:53.286802+0000","last_fullsized":"2024-04-25T20:56:53.286802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:15.588694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285423+0000","last_change":"2024-04-25T20:56:51.273204+0000","last_active":"2024-04-25T20:56:53.285423+0000","last_peered":"2024-04-25T20:56:53.285423+0000","last_clean":"2024-04-25T20:56:53.285423+0000","last_became_active":"2024-04-25T20:56:51.272126+0000","last_became_peered":"2024-04-25T20:56:51.272126+0000","last_unstale":"2024-04-25T20:56:53.285423+0000","last_undegraded":"2024-04-25T20:56:53.285423+0000","last_fullsized":"2024-04-25T20:56:53.285423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:31.684294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284080+0000","last_change":"2024-04-25T20:56:51.270762+0000","last_active":"2024-04-25T20:56:53.284080+0000","last_peered":"2024-04-25T20:56:53.284080+0000","last_clean":"2024-04-25T20:56:53.284080+0000","last_became_active":"2024-04-25T20:56:51.270444+0000","last_became_peered":"2024-04-25T20:56:51.270444+0000","last_unstale":"2024-04-25T20:56:53.284080+0000","last_undegraded":"2024-04-25T20:56:53.284080+0000","last_fullsized":"2024-04-25T20:56:53.284080+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:00.155417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273637+0000","last_change":"2024-04-25T20:56:51.276029+0000","last_active":"2024-04-25T20:56:53.273637+0000","last_peered":"2024-04-25T20:56:53.273637+0000","last_clean":"2024-04-25T20:56:53.273637+0000","last_became_active":"2024-04-25T20:56:51.275272+0000","last_became_peered":"2024-04-25T20:56:51.275272+0000","last_unstale":"2024-04-25T20:56:53.273637+0000","last_undegraded":"2024-04-25T20:56:53.273637+0000","last_fullsized":"2024-04-25T20:56:53.273637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.428444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284911+0000","last_change":"2024-04-25T20:56:51.273769+0000","last_active":"2024-04-25T20:56:53.284911+0000","last_peered":"2024-04-25T20:56:53.284911+0000","last_clean":"2024-04-25T20:56:53.284911+0000","last_became_active":"2024-04-25T20:56:51.273412+0000","last_became_peered":"2024-04-25T20:56:51.273412+0000","last_unstale":"2024-04-25T20:56:53.284911+0000","last_undegraded":"2024-04-25T20:56:53.284911+0000","last_fullsized":"2024-04-25T20:56:53.284911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.324332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288814+0000","last_change":"2024-04-25T20:56:51.277097+0000","last_active":"2024-04-25T20:56:53.288814+0000","last_peered":"2024-04-25T20:56:53.288814+0000","last_clean":"2024-04-25T20:56:53.288814+0000","last_became_active":"2024-04-25T20:56:51.276768+0000","last_became_peered":"2024-04-25T20:56:51.276768+0000","last_unstale":"2024-04-25T20:56:53.288814+0000","last_undegraded":"2024-04-25T20:56:53.288814+0000","last_fullsized":"2024-04-25T20:56:53.288814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:23.276470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285214+0000","last_change":"2024-04-25T20:56:51.279046+0000","last_active":"2024-04-25T20:56:53.285214+0000","last_peered":"2024-04-25T20:56:53.285214+0000","last_clean":"2024-04-25T20:56:53.285214+0000","last_became_active":"2024-04-25T20:56:51.278701+0000","last_became_peered":"2024-04-25T20:56:51.278701+0000","last_unstale":"2024-04-25T20:56:53.285214+0000","last_undegraded":"2024-04-25T20:56:53.285214+0000","last_fullsized":"2024-04-25T20:56:53.285214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.370482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274307+0000","last_change":"2024-04-25T20:56:51.271593+0000","last_active":"2024-04-25T20:56:53.274307+0000","last_peered":"2024-04-25T20:56:53.274307+0000","last_clean":"2024-04-25T20:56:53.274307+0000","last_became_active":"2024-04-25T20:56:51.271224+0000","last_became_peered":"2024-04-25T20:56:51.271224+0000","last_unstale":"2024-04-25T20:56:53.274307+0000","last_undegraded":"2024-04-25T20:56:53.274307+0000","last_fullsized":"2024-04-25T20:56:53.274307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:46.847929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286421+0000","last_change":"2024-04-25T20:56:51.278281+0000","last_active":"2024-04-25T20:56:53.286421+0000","last_peered":"2024-04-25T20:56:53.286421+0000","last_clean":"2024-04-25T20:56:53.286421+0000","last_became_active":"2024-04-25T20:56:51.277947+0000","last_became_peered":"2024-04-25T20:56:51.277947+0000","last_unstale":"2024-04-25T20:56:53.286421+0000","last_undegraded":"2024-04-25T20:56:53.286421+0000","last_fullsized":"2024-04-25T20:56:53.286421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:02.509296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284164+0000","last_change":"2024-04-25T20:56:51.273784+0000","last_active":"2024-04-25T20:56:53.284164+0000","last_peered":"2024-04-25T20:56:53.284164+0000","last_clean":"2024-04-25T20:56:53.284164+0000","last_became_active":"2024-04-25T20:56:51.273489+0000","last_became_peered":"2024-04-25T20:56:51.273489+0000","last_unstale":"2024-04-25T20:56:53.284164+0000","last_undegraded":"2024-04-25T20:56:53.284164+0000","last_fullsized":"2024-04-25T20:56:53.284164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:00.765186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287961+0000","last_change":"2024-04-25T20:56:51.269829+0000","last_active":"2024-04-25T20:56:53.287961+0000","last_peered":"2024-04-25T20:56:53.287961+0000","last_clean":"2024-04-25T20:56:53.287961+0000","last_became_active":"2024-04-25T20:56:51.269446+0000","last_became_peered":"2024-04-25T20:56:51.269446+0000","last_unstale":"2024-04-25T20:56:53.287961+0000","last_undegraded":"2024-04-25T20:56:53.287961+0000","last_fullsized":"2024-04-25T20:56:53.287961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:12.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287557+0000","last_change":"2024-04-25T20:56:51.278117+0000","last_active":"2024-04-25T20:56:53.287557+0000","last_peered":"2024-04-25T20:56:53.287557+0000","last_clean":"2024-04-25T20:56:53.287557+0000","last_became_active":"2024-04-25T20:56:51.277596+0000","last_became_peered":"2024-04-25T20:56:51.277596+0000","last_unstale":"2024-04-25T20:56:53.287557+0000","last_undegraded":"2024-04-25T20:56:53.287557+0000","last_fullsized":"2024-04-25T20:56:53.287557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:28.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287275+0000","last_change":"2024-04-25T20:56:51.281575+0000","last_active":"2024-04-25T20:56:53.287275+0000","last_peered":"2024-04-25T20:56:53.287275+0000","last_clean":"2024-04-25T20:56:53.287275+0000","last_became_active":"2024-04-25T20:56:51.281217+0000","last_became_peered":"2024-04-25T20:56:51.281217+0000","last_unstale":"2024-04-25T20:56:53.287275+0000","last_undegraded":"2024-04-25T20:56:53.287275+0000","last_fullsized":"2024-04-25T20:56:53.287275+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:05.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.292384+0000","last_change":"2024-04-25T20:56:51.284608+0000","last_active":"2024-04-25T20:56:53.292384+0000","last_peered":"2024-04-25T20:56:53.292384+0000","last_clean":"2024-04-25T20:56:53.292384+0000","last_became_active":"2024-04-25T20:56:51.284257+0000","last_became_peered":"2024-04-25T20:56:51.284257+0000","last_unstale":"2024-04-25T20:56:53.292384+0000","last_undegraded":"2024-04-25T20:56:53.292384+0000","last_fullsized":"2024-04-25T20:56:53.292384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:44:59.184192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.272282+0000","last_change":"2024-04-25T20:56:51.276770+0000","last_active":"2024-04-25T20:56:53.272282+0000","last_peered":"2024-04-25T20:56:53.272282+0000","last_clean":"2024-04-25T20:56:53.272282+0000","last_became_active":"2024-04-25T20:56:51.276431+0000","last_became_peered":"2024-04-25T20:56:51.276431+0000","last_unstale":"2024-04-25T20:56:53.272282+0000","last_undegraded":"2024-04-25T20:56:53.272282+0000","last_fullsized":"2024-04-25T20:56:53.272282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:14:25.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273277+0000","last_change":"2024-04-25T20:56:51.283728+0000","last_active":"2024-04-25T20:56:53.273277+0000","last_peered":"2024-04-25T20:56:53.273277+0000","last_clean":"2024-04-25T20:56:53.273277+0000","last_became_active":"2024-04-25T20:56:51.283370+0000","last_became_peered":"2024-04-25T20:56:51.283370+0000","last_unstale":"2024-04-25T20:56:53.273277+0000","last_undegraded":"2024-04-25T20:56:53.273277+0000","last_fullsized":"2024-04-25T20:56:53.273277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:18.042769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.725258+0000","last_change":"2024-04-25T20:56:48.259737+0000","last_active":"2024-04-25T20:57:06.725258+0000","last_peered":"2024-04-25T20:57:06.725258+0000","last_clean":"2024-04-25T20:57:06.725258+0000","last_became_active":"2024-04-25T20:56:48.259442+0000","last_became_peered":"2024-04-25T20:56:48.259442+0000","last_unstale":"2024-04-25T20:57:06.725258+0000","last_undegraded":"2024-04-25T20:57:06.725258+0000","last_fullsized":"2024-04-25T20:57:06.725258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:23.191536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285403+0000","last_change":"2024-04-25T20:56:51.274510+0000","last_active":"2024-04-25T20:56:53.285403+0000","last_peered":"2024-04-25T20:56:53.285403+0000","last_clean":"2024-04-25T20:56:53.285403+0000","last_became_active":"2024-04-25T20:56:51.274137+0000","last_became_peered":"2024-04-25T20:56:51.274137+0000","last_unstale":"2024-04-25T20:56:53.285403+0000","last_undegraded":"2024-04-25T20:56:53.285403+0000","last_fullsized":"2024-04-25T20:56:53.285403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:05.903617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.292596+0000","last_change":"2024-04-25T20:56:48.947850+0000","last_active":"2024-04-25T20:56:53.292596+0000","last_peered":"2024-04-25T20:56:53.292596+0000","last_clean":"2024-04-25T20:56:53.292596+0000","last_became_active":"2024-04-25T20:56:48.946160+0000","last_became_peered":"2024-04-25T20:56:48.946160+0000","last_unstale":"2024-04-25T20:56:53.292596+0000","last_undegraded":"2024-04-25T20:56:53.292596+0000","last_fullsized":"2024-04-25T20:56:53.292596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:32.692390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288569+0000","last_change":"2024-04-25T20:56:51.280478+0000","last_active":"2024-04-25T20:56:53.288569+0000","last_peered":"2024-04-25T20:56:53.288569+0000","last_clean":"2024-04-25T20:56:53.288569+0000","last_became_active":"2024-04-25T20:56:51.280173+0000","last_became_peered":"2024-04-25T20:56:51.280173+0000","last_unstale":"2024-04-25T20:56:53.288569+0000","last_undegraded":"2024-04-25T20:56:53.288569+0000","last_fullsized":"2024-04-25T20:56:53.288569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:48.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.722657+0000","last_change":"2024-04-25T20:56:48.263494+0000","last_active":"2024-04-25T20:57:06.722657+0000","last_peered":"2024-04-25T20:57:06.722657+0000","last_clean":"2024-04-25T20:57:06.722657+0000","last_became_active":"2024-04-25T20:56:48.262726+0000","last_became_peered":"2024-04-25T20:56:48.262726+0000","last_unstale":"2024-04-25T20:57:06.722657+0000","last_undegraded":"2024-04-25T20:57:06.722657+0000","last_fullsized":"2024-04-25T20:57:06.722657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:07.616369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287499+0000","last_change":"2024-04-25T20:56:51.278383+0000","last_active":"2024-04-25T20:56:53.287499+0000","last_peered":"2024-04-25T20:56:53.287499+0000","last_clean":"2024-04-25T20:56:53.287499+0000","last_became_active":"2024-04-25T20:56:51.277603+0000","last_became_peered":"2024-04-25T20:56:51.277603+0000","last_unstale":"2024-04-25T20:56:53.287499+0000","last_undegraded":"2024-04-25T20:56:53.287499+0000","last_fullsized":"2024-04-25T20:56:53.287499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:12.197940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291784+0000","last_change":"2024-04-25T20:56:48.258869+0000","last_active":"2024-04-25T20:56:53.291784+0000","last_peered":"2024-04-25T20:56:53.291784+0000","last_clean":"2024-04-25T20:56:53.291784+0000","last_became_active":"2024-04-25T20:56:48.258414+0000","last_became_peered":"2024-04-25T20:56:48.258414+0000","last_unstale":"2024-04-25T20:56:53.291784+0000","last_undegraded":"2024-04-25T20:56:53.291784+0000","last_fullsized":"2024-04-25T20:56:53.291784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:09.889936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286416+0000","last_change":"2024-04-25T20:56:51.276579+0000","last_active":"2024-04-25T20:56:53.286416+0000","last_peered":"2024-04-25T20:56:53.286416+0000","last_clean":"2024-04-25T20:56:53.286416+0000","last_became_active":"2024-04-25T20:56:51.276252+0000","last_became_peered":"2024-04-25T20:56:51.276252+0000","last_unstale":"2024-04-25T20:56:53.286416+0000","last_undegraded":"2024-04-25T20:56:53.286416+0000","last_fullsized":"2024-04-25T20:56:53.286416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:06.764359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:31.770328+0000","last_change":"2024-04-25T20:56:48.262543+0000","last_active":"2024-04-25T20:57:31.770328+0000","last_peered":"2024-04-25T20:57:31.770328+0000","last_clean":"2024-04-25T20:57:31.770328+0000","last_became_active":"2024-04-25T20:56:48.262086+0000","last_became_peered":"2024-04-25T20:56:48.262086+0000","last_unstale":"2024-04-25T20:57:31.770328+0000","last_undegraded":"2024-04-25T20:57:31.770328+0000","last_fullsized":"2024-04-25T20:57:31.770328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:43.722131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284819+0000","last_change":"2024-04-25T20:56:51.273500+0000","last_active":"2024-04-25T20:56:53.284819+0000","last_peered":"2024-04-25T20:56:53.284819+0000","last_clean":"2024-04-25T20:56:53.284819+0000","last_became_active":"2024-04-25T20:56:51.272882+0000","last_became_peered":"2024-04-25T20:56:51.272882+0000","last_unstale":"2024-04-25T20:56:53.284819+0000","last_undegraded":"2024-04-25T20:56:53.284819+0000","last_fullsized":"2024-04-25T20:56:53.284819+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:05.682258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.723704+0000","last_change":"2024-04-25T20:56:48.255225+0000","last_active":"2024-04-25T20:57:06.723704+0000","last_peered":"2024-04-25T20:57:06.723704+0000","last_clean":"2024-04-25T20:57:06.723704+0000","last_became_active":"2024-04-25T20:56:48.254644+0000","last_became_peered":"2024-04-25T20:56:48.254644+0000","last_unstale":"2024-04-25T20:57:06.723704+0000","last_undegraded":"2024-04-25T20:57:06.723704+0000","last_fullsized":"2024-04-25T20:57:06.723704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:35.724612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286680+0000","last_change":"2024-04-25T20:56:51.270709+0000","last_active":"2024-04-25T20:56:53.286680+0000","last_peered":"2024-04-25T20:56:53.286680+0000","last_clean":"2024-04-25T20:56:53.286680+0000","last_became_active":"2024-04-25T20:56:51.270295+0000","last_became_peered":"2024-04-25T20:56:51.270295+0000","last_unstale":"2024-04-25T20:56:53.286680+0000","last_undegraded":"2024-04-25T20:56:53.286680+0000","last_fullsized":"2024-04-25T20:56:53.286680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:41.998953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.723299+0000","last_change":"2024-04-25T20:56:48.256498+0000","last_active":"2024-04-25T20:57:06.723299+0000","last_peered":"2024-04-25T20:57:06.723299+0000","last_clean":"2024-04-25T20:57:06.723299+0000","last_became_active":"2024-04-25T20:56:48.256140+0000","last_became_peered":"2024-04-25T20:56:48.256140+0000","last_unstale":"2024-04-25T20:57:06.723299+0000","last_undegraded":"2024-04-25T20:57:06.723299+0000","last_fullsized":"2024-04-25T20:57:06.723299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:41.458235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284998+0000","last_change":"2024-04-25T20:56:51.267575+0000","last_active":"2024-04-25T20:56:53.284998+0000","last_peered":"2024-04-25T20:56:53.284998+0000","last_clean":"2024-04-25T20:56:53.284998+0000","last_became_active":"2024-04-25T20:56:51.267265+0000","last_became_peered":"2024-04-25T20:56:51.267265+0000","last_unstale":"2024-04-25T20:56:53.284998+0000","last_undegraded":"2024-04-25T20:56:53.284998+0000","last_fullsized":"2024-04-25T20:56:53.284998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:30.760555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285718+0000","last_change":"2024-04-25T20:56:48.260747+0000","last_active":"2024-04-25T20:56:53.285718+0000","last_peered":"2024-04-25T20:56:53.285718+0000","last_clean":"2024-04-25T20:56:53.285718+0000","last_became_active":"2024-04-25T20:56:48.260339+0000","last_became_peered":"2024-04-25T20:56:48.260339+0000","last_unstale":"2024-04-25T20:56:53.285718+0000","last_undegraded":"2024-04-25T20:56:53.285718+0000","last_fullsized":"2024-04-25T20:56:53.285718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:19.769631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.271980+0000","last_change":"2024-04-25T20:56:51.268365+0000","last_active":"2024-04-25T20:56:53.271980+0000","last_peered":"2024-04-25T20:56:53.271980+0000","last_clean":"2024-04-25T20:56:53.271980+0000","last_became_active":"2024-04-25T20:56:51.268048+0000","last_became_peered":"2024-04-25T20:56:51.268048+0000","last_unstale":"2024-04-25T20:56:53.271980+0000","last_undegraded":"2024-04-25T20:56:53.271980+0000","last_fullsized":"2024-04-25T20:56:53.271980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:31.763147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:33.772461+0000","last_change":"2024-04-25T20:56:48.256113+0000","last_active":"2024-04-25T20:57:33.772461+0000","last_peered":"2024-04-25T20:57:33.772461+0000","last_clean":"2024-04-25T20:57:33.772461+0000","last_became_active":"2024-04-25T20:56:48.255644+0000","last_became_peered":"2024-04-25T20:56:48.255644+0000","last_unstale":"2024-04-25T20:57:33.772461+0000","last_undegraded":"2024-04-25T20:57:33.772461+0000","last_fullsized":"2024-04-25T20:57:33.772461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:12.100099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288156+0000","last_change":"2024-04-25T20:56:51.276990+0000","last_active":"2024-04-25T20:56:53.288156+0000","last_peered":"2024-04-25T20:56:53.288156+0000","last_clean":"2024-04-25T20:56:53.288156+0000","last_became_active":"2024-04-25T20:56:51.276638+0000","last_became_peered":"2024-04-25T20:56:51.276638+0000","last_unstale":"2024-04-25T20:56:53.288156+0000","last_undegraded":"2024-04-25T20:56:53.288156+0000","last_fullsized":"2024-04-25T20:56:53.288156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:06.645278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.713242+0000","last_change":"2024-04-25T20:56:48.263216+0000","last_active":"2024-04-25T20:57:05.713242+0000","last_peered":"2024-04-25T20:57:05.713242+0000","last_clean":"2024-04-25T20:57:05.713242+0000","last_became_active":"2024-04-25T20:56:48.262854+0000","last_became_peered":"2024-04-25T20:56:48.262854+0000","last_unstale":"2024-04-25T20:57:05.713242+0000","last_undegraded":"2024-04-25T20:57:05.713242+0000","last_fullsized":"2024-04-25T20:57:05.713242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:00.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288446+0000","last_change":"2024-04-25T20:56:51.278479+0000","last_active":"2024-04-25T20:56:53.288446+0000","last_peered":"2024-04-25T20:56:53.288446+0000","last_clean":"2024-04-25T20:56:53.288446+0000","last_became_active":"2024-04-25T20:56:51.276285+0000","last_became_peered":"2024-04-25T20:56:51.276285+0000","last_unstale":"2024-04-25T20:56:53.288446+0000","last_undegraded":"2024-04-25T20:56:53.288446+0000","last_fullsized":"2024-04-25T20:56:53.288446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:14.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.712562+0000","last_change":"2024-04-25T20:56:48.946808+0000","last_active":"2024-04-25T20:57:05.712562+0000","last_peered":"2024-04-25T20:57:05.712562+0000","last_clean":"2024-04-25T20:57:05.712562+0000","last_became_active":"2024-04-25T20:56:48.946421+0000","last_became_peered":"2024-04-25T20:56:48.946421+0000","last_unstale":"2024-04-25T20:57:05.712562+0000","last_undegraded":"2024-04-25T20:57:05.712562+0000","last_fullsized":"2024-04-25T20:57:05.712562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:47.763950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288512+0000","last_change":"2024-04-25T20:56:51.274761+0000","last_active":"2024-04-25T20:56:53.288512+0000","last_peered":"2024-04-25T20:56:53.288512+0000","last_clean":"2024-04-25T20:56:53.288512+0000","last_became_active":"2024-04-25T20:56:51.274452+0000","last_became_peered":"2024-04-25T20:56:51.274452+0000","last_unstale":"2024-04-25T20:56:53.288512+0000","last_undegraded":"2024-04-25T20:56:53.288512+0000","last_fullsized":"2024-04-25T20:56:53.288512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:32.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'27","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:34.774058+0000","last_change":"2024-04-25T20:56:48.258982+0000","last_active":"2024-04-25T20:57:34.774058+0000","last_peered":"2024-04-25T20:57:34.774058+0000","last_clean":"2024-04-25T20:57:34.774058+0000","last_became_active":"2024-04-25T20:56:48.258593+0000","last_became_peered":"2024-04-25T20:56:48.258593+0000","last_unstale":"2024-04-25T20:57:34.774058+0000","last_undegraded":"2024-04-25T20:57:34.774058+0000","last_fullsized":"2024-04-25T20:57:34.774058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:20.518202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56090,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":62,"num_read_kb":85,"num_write":28,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285754+0000","last_change":"2024-04-25T20:56:51.276361+0000","last_active":"2024-04-25T20:56:53.285754+0000","last_peered":"2024-04-25T20:56:53.285754+0000","last_clean":"2024-04-25T20:56:53.285754+0000","last_became_active":"2024-04-25T20:56:51.276045+0000","last_became_peered":"2024-04-25T20:56:51.276045+0000","last_unstale":"2024-04-25T20:56:53.285754+0000","last_undegraded":"2024-04-25T20:56:53.285754+0000","last_fullsized":"2024-04-25T20:56:53.285754+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:04.325068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.723933+0000","last_change":"2024-04-25T20:56:48.252358+0000","last_active":"2024-04-25T20:57:06.723933+0000","last_peered":"2024-04-25T20:57:06.723933+0000","last_clean":"2024-04-25T20:57:06.723933+0000","last_became_active":"2024-04-25T20:56:48.252027+0000","last_became_peered":"2024-04-25T20:56:48.252027+0000","last_unstale":"2024-04-25T20:57:06.723933+0000","last_undegraded":"2024-04-25T20:57:06.723933+0000","last_fullsized":"2024-04-25T20:57:06.723933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:03.024487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.283842+0000","last_change":"2024-04-25T20:56:51.279599+0000","last_active":"2024-04-25T20:56:53.283842+0000","last_peered":"2024-04-25T20:56:53.283842+0000","last_clean":"2024-04-25T20:56:53.283842+0000","last_became_active":"2024-04-25T20:56:51.279294+0000","last_became_peered":"2024-04-25T20:56:51.279294+0000","last_unstale":"2024-04-25T20:56:53.283842+0000","last_undegraded":"2024-04-25T20:56:53.283842+0000","last_fullsized":"2024-04-25T20:56:53.283842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.712468+0000","last_change":"2024-04-25T20:56:48.261005+0000","last_active":"2024-04-25T20:57:05.712468+0000","last_peered":"2024-04-25T20:57:05.712468+0000","last_clean":"2024-04-25T20:57:05.712468+0000","last_became_active":"2024-04-25T20:56:48.260543+0000","last_became_peered":"2024-04-25T20:56:48.260543+0000","last_unstale":"2024-04-25T20:57:05.712468+0000","last_undegraded":"2024-04-25T20:57:05.712468+0000","last_fullsized":"2024-04-25T20:57:05.712468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:41:54.326036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274121+0000","last_change":"2024-04-25T20:56:51.271753+0000","last_active":"2024-04-25T20:56:53.274121+0000","last_peered":"2024-04-25T20:56:53.274121+0000","last_clean":"2024-04-25T20:56:53.274121+0000","last_became_active":"2024-04-25T20:56:51.271392+0000","last_became_peered":"2024-04-25T20:56:51.271392+0000","last_unstale":"2024-04-25T20:56:53.274121+0000","last_undegraded":"2024-04-25T20:56:53.274121+0000","last_fullsized":"2024-04-25T20:56:53.274121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:26.605647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:31.770158+0000","last_change":"2024-04-25T20:56:48.260729+0000","last_active":"2024-04-25T20:57:31.770158+0000","last_peered":"2024-04-25T20:57:31.770158+0000","last_clean":"2024-04-25T20:57:31.770158+0000","last_became_active":"2024-04-25T20:56:48.260377+0000","last_became_peered":"2024-04-25T20:56:48.260377+0000","last_unstale":"2024-04-25T20:57:31.770158+0000","last_undegraded":"2024-04-25T20:57:31.770158+0000","last_fullsized":"2024-04-25T20:57:31.770158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:21.582266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273509+0000","last_change":"2024-04-25T20:56:51.275691+0000","last_active":"2024-04-25T20:56:53.273509+0000","last_peered":"2024-04-25T20:56:53.273509+0000","last_clean":"2024-04-25T20:56:53.273509+0000","last_became_active":"2024-04-25T20:56:51.275220+0000","last_became_peered":"2024-04-25T20:56:51.275220+0000","last_unstale":"2024-04-25T20:56:53.273509+0000","last_undegraded":"2024-04-25T20:56:53.273509+0000","last_fullsized":"2024-04-25T20:56:53.273509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:24.367413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.724849+0000","last_change":"2024-04-25T20:56:48.946465+0000","last_active":"2024-04-25T20:57:06.724849+0000","last_peered":"2024-04-25T20:57:06.724849+0000","last_clean":"2024-04-25T20:57:06.724849+0000","last_became_active":"2024-04-25T20:56:48.946093+0000","last_became_peered":"2024-04-25T20:56:48.946093+0000","last_unstale":"2024-04-25T20:57:06.724849+0000","last_undegraded":"2024-04-25T20:57:06.724849+0000","last_fullsized":"2024-04-25T20:57:06.724849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:29.502073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286688+0000","last_change":"2024-04-25T20:56:51.278464+0000","last_active":"2024-04-25T20:56:53.286688+0000","last_peered":"2024-04-25T20:56:53.286688+0000","last_clean":"2024-04-25T20:56:53.286688+0000","last_became_active":"2024-04-25T20:56:51.278119+0000","last_became_peered":"2024-04-25T20:56:51.278119+0000","last_unstale":"2024-04-25T20:56:53.286688+0000","last_undegraded":"2024-04-25T20:56:53.286688+0000","last_fullsized":"2024-04-25T20:56:53.286688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:57:37.224817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:07.718866+0000","last_change":"2024-04-25T20:56:48.255123+0000","last_active":"2024-04-25T20:57:07.718866+0000","last_peered":"2024-04-25T20:57:07.718866+0000","last_clean":"2024-04-25T20:57:07.718866+0000","last_became_active":"2024-04-25T20:56:48.254680+0000","last_became_peered":"2024-04-25T20:56:48.254680+0000","last_unstale":"2024-04-25T20:57:07.718866+0000","last_undegraded":"2024-04-25T20:57:07.718866+0000","last_fullsized":"2024-04-25T20:57:07.718866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:24.089590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291463+0000","last_change":"2024-04-25T20:56:51.283640+0000","last_active":"2024-04-25T20:56:53.291463+0000","last_peered":"2024-04-25T20:56:53.291463+0000","last_clean":"2024-04-25T20:56:53.291463+0000","last_became_active":"2024-04-25T20:56:51.283307+0000","last_became_peered":"2024-04-25T20:56:51.283307+0000","last_unstale":"2024-04-25T20:56:53.291463+0000","last_undegraded":"2024-04-25T20:56:53.291463+0000","last_fullsized":"2024-04-25T20:56:53.291463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:30.087816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.272015+0000","last_change":"2024-04-25T20:56:48.947508+0000","last_active":"2024-04-25T20:56:53.272015+0000","last_peered":"2024-04-25T20:56:53.272015+0000","last_clean":"2024-04-25T20:56:53.272015+0000","last_became_active":"2024-04-25T20:56:48.946582+0000","last_became_peered":"2024-04-25T20:56:48.946582+0000","last_unstale":"2024-04-25T20:56:53.272015+0000","last_undegraded":"2024-04-25T20:56:53.272015+0000","last_fullsized":"2024-04-25T20:56:53.272015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:11.198667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284330+0000","last_change":"2024-04-25T20:56:51.273560+0000","last_active":"2024-04-25T20:56:53.284330+0000","last_peered":"2024-04-25T20:56:53.284330+0000","last_clean":"2024-04-25T20:56:53.284330+0000","last_became_active":"2024-04-25T20:56:51.273182+0000","last_became_peered":"2024-04-25T20:56:51.273182+0000","last_unstale":"2024-04-25T20:56:53.284330+0000","last_undegraded":"2024-04-25T20:56:53.284330+0000","last_fullsized":"2024-04-25T20:56:53.284330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:47.012771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273833+0000","last_change":"2024-04-25T20:56:48.260754+0000","last_active":"2024-04-25T20:56:53.273833+0000","last_peered":"2024-04-25T20:56:53.273833+0000","last_clean":"2024-04-25T20:56:53.273833+0000","last_became_active":"2024-04-25T20:56:48.260420+0000","last_became_peered":"2024-04-25T20:56:48.260420+0000","last_unstale":"2024-04-25T20:56:53.273833+0000","last_undegraded":"2024-04-25T20:56:53.273833+0000","last_fullsized":"2024-04-25T20:56:53.273833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:00.219023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.283925+0000","last_change":"2024-04-25T20:56:51.271944+0000","last_active":"2024-04-25T20:56:53.283925+0000","last_peered":"2024-04-25T20:56:53.283925+0000","last_clean":"2024-04-25T20:56:53.283925+0000","last_became_active":"2024-04-25T20:56:51.270656+0000","last_became_peered":"2024-04-25T20:56:51.270656+0000","last_unstale":"2024-04-25T20:56:53.283925+0000","last_undegraded":"2024-04-25T20:56:53.283925+0000","last_fullsized":"2024-04-25T20:56:53.283925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:00.624888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286455+0000","last_change":"2024-04-25T20:56:51.281207+0000","last_active":"2024-04-25T20:56:53.286455+0000","last_peered":"2024-04-25T20:56:53.286455+0000","last_clean":"2024-04-25T20:56:53.286455+0000","last_became_active":"2024-04-25T20:56:51.280879+0000","last_became_peered":"2024-04-25T20:56:51.280879+0000","last_unstale":"2024-04-25T20:56:53.286455+0000","last_undegraded":"2024-04-25T20:56:53.286455+0000","last_fullsized":"2024-04-25T20:56:53.286455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:01:48.356766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:34.773578+0000","last_change":"2024-04-25T20:56:48.946471+0000","last_active":"2024-04-25T20:57:34.773578+0000","last_peered":"2024-04-25T20:57:34.773578+0000","last_clean":"2024-04-25T20:57:34.773578+0000","last_became_active":"2024-04-25T20:56:48.946029+0000","last_became_peered":"2024-04-25T20:56:48.946029+0000","last_unstale":"2024-04-25T20:57:34.773578+0000","last_undegraded":"2024-04-25T20:57:34.773578+0000","last_fullsized":"2024-04-25T20:57:34.773578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:30:16.400157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285884+0000","last_change":"2024-04-25T20:56:51.278521+0000","last_active":"2024-04-25T20:56:53.285884+0000","last_peered":"2024-04-25T20:56:53.285884+0000","last_clean":"2024-04-25T20:56:53.285884+0000","last_became_active":"2024-04-25T20:56:51.278224+0000","last_became_peered":"2024-04-25T20:56:51.278224+0000","last_unstale":"2024-04-25T20:56:53.285884+0000","last_undegraded":"2024-04-25T20:56:53.285884+0000","last_fullsized":"2024-04-25T20:56:53.285884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:57.211010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:08.731857+0000","last_change":"2024-04-25T20:56:48.258479+0000","last_active":"2024-04-25T20:57:08.731857+0000","last_peered":"2024-04-25T20:57:08.731857+0000","last_clean":"2024-04-25T20:57:08.731857+0000","last_became_active":"2024-04-25T20:56:48.257951+0000","last_became_peered":"2024-04-25T20:56:48.257951+0000","last_unstale":"2024-04-25T20:57:08.731857+0000","last_undegraded":"2024-04-25T20:57:08.731857+0000","last_fullsized":"2024-04-25T20:57:08.731857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:13.812201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273477+0000","last_change":"2024-04-25T20:55:28.701357+0000","last_active":"2024-04-25T20:56:53.273477+0000","last_peered":"2024-04-25T20:56:53.273477+0000","last_clean":"2024-04-25T20:56:53.273477+0000","last_became_active":"2024-04-25T20:55:28.700906+0000","last_became_peered":"2024-04-25T20:55:28.700906+0000","last_unstale":"2024-04-25T20:56:53.273477+0000","last_undegraded":"2024-04-25T20:56:53.273477+0000","last_fullsized":"2024-04-25T20:56:53.273477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_clean_scrub_stamp":"2024-04-25T20:52:10.334323+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:37:35.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.283530+0000","last_change":"2024-04-25T20:56:51.275249+0000","last_active":"2024-04-25T20:56:53.283530+0000","last_peered":"2024-04-25T20:56:53.283530+0000","last_clean":"2024-04-25T20:56:53.283530+0000","last_became_active":"2024-04-25T20:56:51.274938+0000","last_became_peered":"2024-04-25T20:56:51.274938+0000","last_unstale":"2024-04-25T20:56:53.283530+0000","last_undegraded":"2024-04-25T20:56:53.283530+0000","last_fullsized":"2024-04-25T20:56:53.283530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:53.933324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.723812+0000","last_change":"2024-04-25T20:56:48.259357+0000","last_active":"2024-04-25T20:57:06.723812+0000","last_peered":"2024-04-25T20:57:06.723812+0000","last_clean":"2024-04-25T20:57:06.723812+0000","last_became_active":"2024-04-25T20:56:48.258840+0000","last_became_peered":"2024-04-25T20:56:48.258840+0000","last_unstale":"2024-04-25T20:57:06.723812+0000","last_undegraded":"2024-04-25T20:57:06.723812+0000","last_fullsized":"2024-04-25T20:57:06.723812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:30.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.712232+0000","last_change":"2024-04-25T20:56:48.946498+0000","last_active":"2024-04-25T20:57:05.712232+0000","last_peered":"2024-04-25T20:57:05.712232+0000","last_clean":"2024-04-25T20:57:05.712232+0000","last_became_active":"2024-04-25T20:56:48.946124+0000","last_became_peered":"2024-04-25T20:56:48.946124+0000","last_unstale":"2024-04-25T20:57:05.712232+0000","last_undegraded":"2024-04-25T20:57:05.712232+0000","last_fullsized":"2024-04-25T20:57:05.712232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:08.545574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284086+0000","last_change":"2024-04-25T20:56:51.276093+0000","last_active":"2024-04-25T20:56:53.284086+0000","last_peered":"2024-04-25T20:56:53.284086+0000","last_clean":"2024-04-25T20:56:53.284086+0000","last_became_active":"2024-04-25T20:56:51.275771+0000","last_became_peered":"2024-04-25T20:56:51.275771+0000","last_unstale":"2024-04-25T20:56:53.284086+0000","last_undegraded":"2024-04-25T20:56:53.284086+0000","last_fullsized":"2024-04-25T20:56:53.284086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:15.159208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:30.287833+0000","last_change":"2024-04-25T20:56:48.251519+0000","last_active":"2024-04-25T20:57:30.287833+0000","last_peered":"2024-04-25T20:57:30.287833+0000","last_clean":"2024-04-25T20:57:30.287833+0000","last_became_active":"2024-04-25T20:56:48.251071+0000","last_became_peered":"2024-04-25T20:56:48.251071+0000","last_unstale":"2024-04-25T20:57:30.287833+0000","last_undegraded":"2024-04-25T20:57:30.287833+0000","last_fullsized":"2024-04-25T20:57:30.287833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:24.583520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285514+0000","last_change":"2024-04-25T20:56:51.275461+0000","last_active":"2024-04-25T20:56:53.285514+0000","last_peered":"2024-04-25T20:56:53.285514+0000","last_clean":"2024-04-25T20:56:53.285514+0000","last_became_active":"2024-04-25T20:56:51.275087+0000","last_became_peered":"2024-04-25T20:56:51.275087+0000","last_unstale":"2024-04-25T20:56:53.285514+0000","last_undegraded":"2024-04-25T20:56:53.285514+0000","last_fullsized":"2024-04-25T20:56:53.285514+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:01:02.514490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.711481+0000","last_change":"2024-04-25T20:56:49.031177+0000","last_active":"2024-04-25T20:57:05.711481+0000","last_peered":"2024-04-25T20:57:05.711481+0000","last_clean":"2024-04-25T20:57:05.711481+0000","last_became_active":"2024-04-25T20:56:49.030697+0000","last_became_peered":"2024-04-25T20:56:49.030697+0000","last_unstale":"2024-04-25T20:57:05.711481+0000","last_undegraded":"2024-04-25T20:57:05.711481+0000","last_fullsized":"2024-04-25T20:57:05.711481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:13.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288295+0000","last_change":"2024-04-25T20:56:51.271157+0000","last_active":"2024-04-25T20:56:53.288295+0000","last_peered":"2024-04-25T20:56:53.288295+0000","last_clean":"2024-04-25T20:56:53.288295+0000","last_became_active":"2024-04-25T20:56:51.270795+0000","last_became_peered":"2024-04-25T20:56:51.270795+0000","last_unstale":"2024-04-25T20:56:53.288295+0000","last_undegraded":"2024-04-25T20:56:53.288295+0000","last_fullsized":"2024-04-25T20:56:53.288295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:36.741663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.713214+0000","last_change":"2024-04-25T20:56:48.946689+0000","last_active":"2024-04-25T20:57:05.713214+0000","last_peered":"2024-04-25T20:57:05.713214+0000","last_clean":"2024-04-25T20:57:05.713214+0000","last_became_active":"2024-04-25T20:56:48.946229+0000","last_became_peered":"2024-04-25T20:56:48.946229+0000","last_unstale":"2024-04-25T20:57:05.713214+0000","last_undegraded":"2024-04-25T20:57:05.713214+0000","last_fullsized":"2024-04-25T20:57:05.713214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:04.120197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284722+0000","last_change":"2024-04-25T20:56:51.275657+0000","last_active":"2024-04-25T20:56:53.284722+0000","last_peered":"2024-04-25T20:56:53.284722+0000","last_clean":"2024-04-25T20:56:53.284722+0000","last_became_active":"2024-04-25T20:56:51.275237+0000","last_became_peered":"2024-04-25T20:56:51.275237+0000","last_unstale":"2024-04-25T20:56:53.284722+0000","last_undegraded":"2024-04-25T20:56:53.284722+0000","last_fullsized":"2024-04-25T20:56:53.284722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:39.778340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.712632+0000","last_change":"2024-04-25T20:56:48.251882+0000","last_active":"2024-04-25T20:57:05.712632+0000","last_peered":"2024-04-25T20:57:05.712632+0000","last_clean":"2024-04-25T20:57:05.712632+0000","last_became_active":"2024-04-25T20:56:48.251571+0000","last_became_peered":"2024-04-25T20:56:48.251571+0000","last_unstale":"2024-04-25T20:57:05.712632+0000","last_undegraded":"2024-04-25T20:57:05.712632+0000","last_fullsized":"2024-04-25T20:57:05.712632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:51.769332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291952+0000","last_change":"2024-04-25T20:56:51.280648+0000","last_active":"2024-04-25T20:56:53.291952+0000","last_peered":"2024-04-25T20:56:53.291952+0000","last_clean":"2024-04-25T20:56:53.291952+0000","last_became_active":"2024-04-25T20:56:51.280232+0000","last_became_peered":"2024-04-25T20:56:51.280232+0000","last_unstale":"2024-04-25T20:56:53.291952+0000","last_undegraded":"2024-04-25T20:56:53.291952+0000","last_fullsized":"2024-04-25T20:56:53.291952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:12.375519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273407+0000","last_change":"2024-04-25T20:56:48.251976+0000","last_active":"2024-04-25T20:56:53.273407+0000","last_peered":"2024-04-25T20:56:53.273407+0000","last_clean":"2024-04-25T20:56:53.273407+0000","last_became_active":"2024-04-25T20:56:48.251549+0000","last_became_peered":"2024-04-25T20:56:48.251549+0000","last_unstale":"2024-04-25T20:56:53.273407+0000","last_undegraded":"2024-04-25T20:56:53.273407+0000","last_fullsized":"2024-04-25T20:56:53.273407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:27.413462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285162+0000","last_change":"2024-04-25T20:56:51.277166+0000","last_active":"2024-04-25T20:56:53.285162+0000","last_peered":"2024-04-25T20:56:53.285162+0000","last_clean":"2024-04-25T20:56:53.285162+0000","last_became_active":"2024-04-25T20:56:51.276840+0000","last_became_peered":"2024-04-25T20:56:51.276840+0000","last_unstale":"2024-04-25T20:56:53.285162+0000","last_undegraded":"2024-04-25T20:56:53.285162+0000","last_fullsized":"2024-04-25T20:56:53.285162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:52.756660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:05.713394+0000","last_change":"2024-04-25T20:56:48.263426+0000","last_active":"2024-04-25T20:57:05.713394+0000","last_peered":"2024-04-25T20:57:05.713394+0000","last_clean":"2024-04-25T20:57:05.713394+0000","last_became_active":"2024-04-25T20:56:48.262832+0000","last_became_peered":"2024-04-25T20:56:48.262832+0000","last_unstale":"2024-04-25T20:57:05.713394+0000","last_undegraded":"2024-04-25T20:57:05.713394+0000","last_fullsized":"2024-04-25T20:57:05.713394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:09.362161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274265+0000","last_change":"2024-04-25T20:56:51.274267+0000","last_active":"2024-04-25T20:56:53.274265+0000","last_peered":"2024-04-25T20:56:53.274265+0000","last_clean":"2024-04-25T20:56:53.274265+0000","last_became_active":"2024-04-25T20:56:51.273807+0000","last_became_peered":"2024-04-25T20:56:51.273807+0000","last_unstale":"2024-04-25T20:56:53.274265+0000","last_undegraded":"2024-04-25T20:56:53.274265+0000","last_fullsized":"2024-04-25T20:56:53.274265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:57:06.724101+0000","last_change":"2024-04-25T20:56:49.031011+0000","last_active":"2024-04-25T20:57:06.724101+0000","last_peered":"2024-04-25T20:57:06.724101+0000","last_clean":"2024-04-25T20:57:06.724101+0000","last_became_active":"2024-04-25T20:56:49.030522+0000","last_became_peered":"2024-04-25T20:56:49.030522+0000","last_unstale":"2024-04-25T20:57:06.724101+0000","last_undegraded":"2024-04-25T20:57:06.724101+0000","last_fullsized":"2024-04-25T20:57:06.724101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:00.915454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274301+0000","last_change":"2024-04-25T20:56:51.276989+0000","last_active":"2024-04-25T20:56:53.274301+0000","last_peered":"2024-04-25T20:56:53.274301+0000","last_clean":"2024-04-25T20:56:53.274301+0000","last_became_active":"2024-04-25T20:56:51.275417+0000","last_became_peered":"2024-04-25T20:56:51.275417+0000","last_unstale":"2024-04-25T20:56:53.274301+0000","last_undegraded":"2024-04-25T20:56:53.274301+0000","last_fullsized":"2024-04-25T20:56:53.274301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:27.516062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.290382+0000","last_change":"2024-04-25T20:56:48.253713+0000","last_active":"2024-04-25T20:56:53.290382+0000","last_peered":"2024-04-25T20:56:53.290382+0000","last_clean":"2024-04-25T20:56:53.290382+0000","last_became_active":"2024-04-25T20:56:48.253215+0000","last_became_peered":"2024-04-25T20:56:48.253215+0000","last_unstale":"2024-04-25T20:56:53.290382+0000","last_undegraded":"2024-04-25T20:56:53.290382+0000","last_fullsized":"2024-04-25T20:56:53.290382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:57.946840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:54.481839+0000","last_change":"2024-04-25T20:56:51.274691+0000","last_active":"2024-04-25T20:56:54.481839+0000","last_peered":"2024-04-25T20:56:54.481839+0000","last_clean":"2024-04-25T20:56:54.481839+0000","last_became_active":"2024-04-25T20:56:51.274350+0000","last_became_peered":"2024-04-25T20:56:51.274350+0000","last_unstale":"2024-04-25T20:56:54.481839+0000","last_undegraded":"2024-04-25T20:56:54.481839+0000","last_fullsized":"2024-04-25T20:56:54.481839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:52.375409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286977+0000","last_change":"2024-04-25T20:56:48.251616+0000","last_active":"2024-04-25T20:56:53.286977+0000","last_peered":"2024-04-25T20:56:53.286977+0000","last_clean":"2024-04-25T20:56:53.286977+0000","last_became_active":"2024-04-25T20:56:48.251248+0000","last_became_peered":"2024-04-25T20:56:48.251248+0000","last_unstale":"2024-04-25T20:56:53.286977+0000","last_undegraded":"2024-04-25T20:56:53.286977+0000","last_fullsized":"2024-04-25T20:56:53.286977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:51.032356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.274201+0000","last_change":"2024-04-25T20:56:51.278372+0000","last_active":"2024-04-25T20:56:53.274201+0000","last_peered":"2024-04-25T20:56:53.274201+0000","last_clean":"2024-04-25T20:56:53.274201+0000","last_became_active":"2024-04-25T20:56:51.277989+0000","last_became_peered":"2024-04-25T20:56:51.277989+0000","last_unstale":"2024-04-25T20:56:53.274201+0000","last_undegraded":"2024-04-25T20:56:53.274201+0000","last_fullsized":"2024-04-25T20:56:53.274201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.017713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.292018+0000","last_change":"2024-04-25T20:56:51.276270+0000","last_active":"2024-04-25T20:56:53.292018+0000","last_peered":"2024-04-25T20:56:53.292018+0000","last_clean":"2024-04-25T20:56:53.292018+0000","last_became_active":"2024-04-25T20:56:51.275948+0000","last_became_peered":"2024-04-25T20:56:51.275948+0000","last_unstale":"2024-04-25T20:56:53.292018+0000","last_undegraded":"2024-04-25T20:56:53.292018+0000","last_fullsized":"2024-04-25T20:56:53.292018+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:33.548439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.272969+0000","last_change":"2024-04-25T20:56:51.283931+0000","last_active":"2024-04-25T20:56:53.272969+0000","last_peered":"2024-04-25T20:56:53.272969+0000","last_clean":"2024-04-25T20:56:53.272969+0000","last_became_active":"2024-04-25T20:56:51.283120+0000","last_became_peered":"2024-04-25T20:56:51.283120+0000","last_unstale":"2024-04-25T20:56:53.272969+0000","last_undegraded":"2024-04-25T20:56:53.272969+0000","last_fullsized":"2024-04-25T20:56:53.272969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:26.676750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.272910+0000","last_change":"2024-04-25T20:56:51.286013+0000","last_active":"2024-04-25T20:56:53.272910+0000","last_peered":"2024-04-25T20:56:53.272910+0000","last_clean":"2024-04-25T20:56:53.272910+0000","last_became_active":"2024-04-25T20:56:51.285645+0000","last_became_peered":"2024-04-25T20:56:51.285645+0000","last_unstale":"2024-04-25T20:56:53.272910+0000","last_undegraded":"2024-04-25T20:56:53.272910+0000","last_fullsized":"2024-04-25T20:56:53.272910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:51.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.287881+0000","last_change":"2024-04-25T20:56:51.272567+0000","last_active":"2024-04-25T20:56:53.287881+0000","last_peered":"2024-04-25T20:56:53.287881+0000","last_clean":"2024-04-25T20:56:53.287881+0000","last_became_active":"2024-04-25T20:56:51.271427+0000","last_became_peered":"2024-04-25T20:56:51.271427+0000","last_unstale":"2024-04-25T20:56:53.287881+0000","last_undegraded":"2024-04-25T20:56:53.287881+0000","last_fullsized":"2024-04-25T20:56:53.287881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:19.945982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.284492+0000","last_change":"2024-04-25T20:56:51.272448+0000","last_active":"2024-04-25T20:56:53.284492+0000","last_peered":"2024-04-25T20:56:53.284492+0000","last_clean":"2024-04-25T20:56:53.284492+0000","last_became_active":"2024-04-25T20:56:51.271965+0000","last_became_peered":"2024-04-25T20:56:51.271965+0000","last_unstale":"2024-04-25T20:56:53.284492+0000","last_undegraded":"2024-04-25T20:56:53.284492+0000","last_fullsized":"2024-04-25T20:56:53.284492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:03.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291752+0000","last_change":"2024-04-25T20:56:51.280260+0000","last_active":"2024-04-25T20:56:53.291752+0000","last_peered":"2024-04-25T20:56:53.291752+0000","last_clean":"2024-04-25T20:56:53.291752+0000","last_became_active":"2024-04-25T20:56:51.279864+0000","last_became_peered":"2024-04-25T20:56:51.279864+0000","last_unstale":"2024-04-25T20:56:53.291752+0000","last_undegraded":"2024-04-25T20:56:53.291752+0000","last_fullsized":"2024-04-25T20:56:53.291752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.136824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.273458+0000","last_change":"2024-04-25T20:56:51.279041+0000","last_active":"2024-04-25T20:56:53.273458+0000","last_peered":"2024-04-25T20:56:53.273458+0000","last_clean":"2024-04-25T20:56:53.273458+0000","last_became_active":"2024-04-25T20:56:51.278302+0000","last_became_peered":"2024-04-25T20:56:51.278302+0000","last_unstale":"2024-04-25T20:56:53.273458+0000","last_undegraded":"2024-04-25T20:56:53.273458+0000","last_fullsized":"2024-04-25T20:56:53.273458+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:02.510919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285280+0000","last_change":"2024-04-25T20:56:51.275618+0000","last_active":"2024-04-25T20:56:53.285280+0000","last_peered":"2024-04-25T20:56:53.285280+0000","last_clean":"2024-04-25T20:56:53.285280+0000","last_became_active":"2024-04-25T20:56:51.275249+0000","last_became_peered":"2024-04-25T20:56:51.275249+0000","last_unstale":"2024-04-25T20:56:53.285280+0000","last_undegraded":"2024-04-25T20:56:53.285280+0000","last_fullsized":"2024-04-25T20:56:53.285280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:01.793083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.285925+0000","last_change":"2024-04-25T20:56:51.275868+0000","last_active":"2024-04-25T20:56:53.285925+0000","last_peered":"2024-04-25T20:56:53.285925+0000","last_clean":"2024-04-25T20:56:53.285925+0000","last_became_active":"2024-04-25T20:56:51.275544+0000","last_became_peered":"2024-04-25T20:56:51.275544+0000","last_unstale":"2024-04-25T20:56:53.285925+0000","last_undegraded":"2024-04-25T20:56:53.285925+0000","last_fullsized":"2024-04-25T20:56:53.285925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:31.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.286522+0000","last_change":"2024-04-25T20:56:51.274963+0000","last_active":"2024-04-25T20:56:53.286522+0000","last_peered":"2024-04-25T20:56:53.286522+0000","last_clean":"2024-04-25T20:56:53.286522+0000","last_became_active":"2024-04-25T20:56:51.274597+0000","last_became_peered":"2024-04-25T20:56:51.274597+0000","last_unstale":"2024-04-25T20:56:53.286522+0000","last_undegraded":"2024-04-25T20:56:53.286522+0000","last_fullsized":"2024-04-25T20:56:53.286522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:14.464560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.288511+0000","last_change":"2024-04-25T20:56:51.279602+0000","last_active":"2024-04-25T20:56:53.288511+0000","last_peered":"2024-04-25T20:56:53.288511+0000","last_clean":"2024-04-25T20:56:53.288511+0000","last_became_active":"2024-04-25T20:56:51.279221+0000","last_became_peered":"2024-04-25T20:56:51.279221+0000","last_unstale":"2024-04-25T20:56:53.288511+0000","last_undegraded":"2024-04-25T20:56:53.288511+0000","last_fullsized":"2024-04-25T20:56:53.288511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:26.960492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.283988+0000","last_change":"2024-04-25T20:56:51.280465+0000","last_active":"2024-04-25T20:56:53.283988+0000","last_peered":"2024-04-25T20:56:53.283988+0000","last_clean":"2024-04-25T20:56:53.283988+0000","last_became_active":"2024-04-25T20:56:51.280168+0000","last_became_peered":"2024-04-25T20:56:51.280168+0000","last_unstale":"2024-04-25T20:56:53.283988+0000","last_undegraded":"2024-04-25T20:56:53.283988+0000","last_fullsized":"2024-04-25T20:56:53.283988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.211113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T20:56:53.291065+0000","last_change":"2024-04-25T20:56:51.283084+0000","last_active":"2024-04-25T20:56:53.291065+0000","last_peered":"2024-04-25T20:56:53.291065+0000","last_clean":"2024-04-25T20:56:53.291065+0000","last_became_active":"2024-04-25T20:56:51.282692+0000","last_became_peered":"2024-04-25T20:56:51.282692+0000","last_unstale":"2024-04-25T20:56:53.291065+0000","last_undegraded":"2024-04-25T20:56:53.291065+0000","last_fullsized":"2024-04-25T20:56:53.291065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:13:27.527112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60482,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":187,"num_read_kb":160,"num_write":164,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":181446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":467042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579934,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1658880,"data_stored":1115004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":467278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037799,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":467198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1059519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462961,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40444,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704708,"statfs":{"total":95995035648,"available":95953620992,"internally_reserved":0,"allocated":970752,"data_stored":466667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":522550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":467938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":524291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":466978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":467495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724598,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1058812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:57:39.223 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T20:57:39.224 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T20:57:39.224 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T20:57:39.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:38 smithi092 ceph-mon[32524]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T20:57:39.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:38 smithi092 ceph-mon[32524]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:39.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:38 smithi186 ceph-mon[32529]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T20:57:39.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:38 smithi186 ceph-mon[32529]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 -- 172.21.15.47:0/3000949599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a001035b0 msgr2=0x7f3a001059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 --2- 172.21.15.47:0/3000949599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a001035b0 0x7f3a001059a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009990 tx=0x7f39f0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 -- 172.21.15.47:0/3000949599 shutdown_connections 2024-04-25T20:57:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 --2- 172.21.15.47:0/3000949599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a001035b0 0x7f3a001059a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 --2- 172.21.15.47:0/3000949599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a00108810 0x7f3a0010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 --2- 172.21.15.47:0/3000949599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00105ee0 0x7f3a001082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 -- 172.21.15.47:0/3000949599 >> 172.21.15.47:0/3000949599 conn(0x7f3a000fcf60 msgr2=0x7f3a000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 -- 172.21.15.47:0/3000949599 shutdown_connections 2024-04-25T20:57:41.191 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:41 smithi047 ceph-mon[29195]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.189+0000 7f3a048b9700 1 -- 172.21.15.47:0/3000949599 wait complete. 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.190+0000 7f3a048b9700 1 Processor -- start 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.190+0000 7f3a048b9700 1 -- start start 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.190+0000 7f3a048b9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a001035b0 0x7f3a00102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.190+0000 7f3a048b9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00105ee0 0x7f3a00101560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.190+0000 7f3a048b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a00108810 0x7f3a00101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.190+0000 7f3a048b9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0010e5e0 con 0x7f3a00108810 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.190+0000 7f3a048b9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0010e460 con 0x7f3a00105ee0 2024-04-25T20:57:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.190+0000 7f3a048b9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0010e760 con 0x7f3a001035b0 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a00108810 0x7f3a00101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a00108810 0x7f3a00101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32888/0 (socket says 172.21.15.47:32888) 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 -- 172.21.15.47:0/760667728 learned_addr learned my addr 172.21.15.47:0/760667728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fe59c700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a001035b0 0x7f3a00102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fdd9b700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00105ee0 0x7f3a00101560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 -- 172.21.15.47:0/760667728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a001035b0 msgr2=0x7f3a00102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a001035b0 0x7f3a00102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 -- 172.21.15.47:0/760667728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00105ee0 msgr2=0x7f3a00101560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00105ee0 0x7f3a00101560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 -- 172.21.15.47:0/760667728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a00102350 con 0x7f3a00108810 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fdd9b700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00105ee0 0x7f3a00101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39fed9d700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a00108810 0x7f3a00101aa0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39f800ada0 tx=0x7f39f800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f39ef7fe700 1 -- 172.21.15.47:0/760667728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8017020 con 0x7f3a00108810 2024-04-25T20:57:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.191+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a001a2d30 con 0x7f3a00108810 2024-04-25T20:57:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.192+0000 7f39ef7fe700 1 -- 172.21.15.47:0/760667728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f800ce30 con 0x7f3a00108810 2024-04-25T20:57:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.192+0000 7f39ef7fe700 1 -- 172.21.15.47:0/760667728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8005310 con 0x7f3a00108810 2024-04-25T20:57:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.192+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a001a3290 con 0x7f3a00108810 2024-04-25T20:57:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.193+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a00050f80 con 0x7f3a00108810 2024-04-25T20:57:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.194+0000 7f39ef7fe700 1 -- 172.21.15.47:0/760667728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f39f800c980 con 0x7f3a00108810 2024-04-25T20:57:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.194+0000 7f39ef7fe700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f39e80a3d80 0x7f39e80a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.194+0000 7f39fe59c700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f39e80a3d80 0x7f39e80a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.194+0000 7f39ef7fe700 1 -- 172.21.15.47:0/760667728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f39f80cfa20 con 0x7f3a00108810 2024-04-25T20:57:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.195+0000 7f39fe59c700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f39e80a3d80 0x7f39e80a6240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39f000b5c0 tx=0x7f39f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.199+0000 7f39ef7fe700 1 -- 172.21.15.47:0/760667728 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39f8021020 con 0x7f3a00108810 2024-04-25T20:57:41.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:41 smithi092 ceph-mon[32524]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T20:57:41.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:41 smithi186 ceph-mon[32529]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T20:57:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.501+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3a0002e0f0 con 0x7f3a00108810 2024-04-25T20:57:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.503+0000 7f39ef7fe700 1 -- 172.21.15.47:0/760667728 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f3a0002e0f0 con 0x7f3a00108810 2024-04-25T20:57:41.504 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-25T20:57:41.504 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDFwypmyd38HRAAD5/rNsdjN26HNXg0q3vOBg== 2024-04-25T20:57:41.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f39e80a3d80 msgr2=0x7f39e80a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f39e80a3d80 0x7f39e80a6240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39f000b5c0 tx=0x7f39f0005fb0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a00108810 msgr2=0x7f3a00101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a00108810 0x7f3a00101aa0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39f800ada0 tx=0x7f39f800c330 comp rx=0 tx=0).stop 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 shutdown_connections 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f39e80a3d80 0x7f39e80a6240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a00108810 0x7f3a00101aa0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a001035b0 0x7f3a00102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 --2- 172.21.15.47:0/760667728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00105ee0 0x7f3a00101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 >> 172.21.15.47:0/760667728 conn(0x7f3a000fcf60 msgr2=0x7f3a00103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 shutdown_connections 2024-04-25T20:57:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:41.506+0000 7f3a048b9700 1 -- 172.21.15.47:0/760667728 wait complete. 2024-04-25T20:57:42.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:42 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/760667728' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T20:57:42.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:42 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/760667728' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T20:57:42.168 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:57:42.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T20:57:42.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T20:57:42.205 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T20:57:42.206 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T20:57:42.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T20:57:42.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:42 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/760667728' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T20:57:42.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:42 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/760667728' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T20:57:42.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:42 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/760667728' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T20:57:42.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:42 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/760667728' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T20:57:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:43.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:43 smithi047 ceph-mon[29195]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T20:57:43.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:43 smithi092 ceph-mon[32524]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T20:57:43.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:43 smithi186 ceph-mon[32529]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T20:57:44.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/2076046195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc1018c0 msgr2=0x7fb6cc101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:44.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/2076046195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc1018c0 0x7fb6cc101d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009990 tx=0x7fb6c0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/2076046195 shutdown_connections 2024-04-25T20:57:44.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/2076046195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc102280 0x7fb6cc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/2076046195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6cc1078c0 0x7fb6cc107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/2076046195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc1018c0 0x7fb6cc101d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/2076046195 >> 172.21.15.47:0/2076046195 conn(0x7fb6cc0fd3d0 msgr2=0x7fb6cc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:44.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/2076046195 shutdown_connections 2024-04-25T20:57:44.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.060+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/2076046195 wait complete. 2024-04-25T20:57:44.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6d1f9d700 1 Processor -- start 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6d1f9d700 1 -- start start 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6d1f9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6cc1018c0 0x7fb6cc0fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6d1f9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc102280 0x7fb6cc0fde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6d1f9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc1078c0 0x7fb6cc0fe370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6d1f9d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc111aa0 con 0x7fb6cc102280 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6d1f9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc111920 con 0x7fb6cc1078c0 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6d1f9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc111c20 con 0x7fb6cc1018c0 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6cbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc102280 0x7fb6cc0fde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6cbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc102280 0x7fb6cc0fde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45030/0 (socket says 172.21.15.47:45030) 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.061+0000 7fb6cbfff700 1 -- 172.21.15.47:0/1713545521 learned_addr learned my addr 172.21.15.47:0/1713545521 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6d179c700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc1078c0 0x7fb6cc0fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6d0f9b700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6cc1018c0 0x7fb6cc0fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6cbfff700 1 -- 172.21.15.47:0/1713545521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6cc1018c0 msgr2=0x7fb6cc0fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6cbfff700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6cc1018c0 0x7fb6cc0fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6cbfff700 1 -- 172.21.15.47:0/1713545521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc1078c0 msgr2=0x7fb6cc0fe370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6cbfff700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc1078c0 0x7fb6cc0fe370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6cbfff700 1 -- 172.21.15.47:0/1713545521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc1a7530 con 0x7fb6cc102280 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6d0f9b700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6cc1018c0 0x7fb6cc0fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6d179c700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc1078c0 0x7fb6cc0fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6cbfff700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc102280 0x7fb6cc0fde30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00057c0 tx=0x7fb6c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6c9ffb700 1 -- 172.21.15.47:0/1713545521 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0027070 con 0x7fb6cc102280 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6c9ffb700 1 -- 172.21.15.47:0/1713545521 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6c0004020 con 0x7fb6cc102280 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6cc1a77c0 con 0x7fb6cc102280 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6c9ffb700 1 -- 172.21.15.47:0/1713545521 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0022bf0 con 0x7fb6cc102280 2024-04-25T20:57:44.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.062+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6cc1a7cf0 con 0x7fb6cc102280 2024-04-25T20:57:44.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.063+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6cc069730 con 0x7fb6cc102280 2024-04-25T20:57:44.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.065+0000 7fb6c9ffb700 1 -- 172.21.15.47:0/1713545521 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb6c002a070 con 0x7fb6cc102280 2024-04-25T20:57:44.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.065+0000 7fb6c9ffb700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb6b40a3d50 0x7fb6b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:44.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.065+0000 7fb6d0f9b700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb6b40a3d50 0x7fb6b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:44.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.065+0000 7fb6c9ffb700 1 -- 172.21.15.47:0/1713545521 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fb6c00d1a60 con 0x7fb6cc102280 2024-04-25T20:57:44.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.066+0000 7fb6d0f9b700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb6b40a3d50 0x7fb6b40a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00baa0 tx=0x7fb6bc005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:44.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.069+0000 7fb6c9ffb700 1 -- 172.21.15.47:0/1713545521 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6c0031020 con 0x7fb6cc102280 2024-04-25T20:57:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.338+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb6cc0fee20 con 0x7fb6cc102280 2024-04-25T20:57:44.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.343+0000 7fb6c9ffb700 1 -- 172.21.15.47:0/1713545521 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264994 (secure 0 0 0) 0x7fb6c0013070 con 0x7fb6cc102280 2024-04-25T20:57:44.348 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb6b40a3d50 msgr2=0x7fb6b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb6b40a3d50 0x7fb6b40a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00baa0 tx=0x7fb6bc005db0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc102280 msgr2=0x7fb6cc0fde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc102280 0x7fb6cc0fde30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00057c0 tx=0x7fb6c00057f0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 shutdown_connections 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb6b40a3d50 0x7fb6b40a6210 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc1078c0 0x7fb6cc0fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6cc1018c0 0x7fb6cc0fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 --2- 172.21.15.47:0/1713545521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6cc102280 0x7fb6cc0fde30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:44.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 >> 172.21.15.47:0/1713545521 conn(0x7fb6cc0fd3d0 msgr2=0x7fb6cc06b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:44.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 shutdown_connections 2024-04-25T20:57:44.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:44.348+0000 7fb6d1f9d700 1 -- 172.21.15.47:0/1713545521 wait complete. 2024-04-25T20:57:44.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:44 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1713545521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T20:57:44.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:44 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1713545521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T20:57:44.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:44 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1713545521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T20:57:45.058 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":253800519},{"type":"v1","addr":"172.21.15.47:6801","nonce":253800519}]},"active_addr":"172.21.15.47:6801/253800519","active_change":"2024-04-25T20:49:45.811251+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.47:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3318702222}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1752629947}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":831854467}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2517449296}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1451525612}]}]} 2024-04-25T20:57:45.064 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T20:57:45.065 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T20:57:45.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:57:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:45.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:45 smithi047 ceph-mon[29195]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 681 B/s wr, 6 op/s 2024-04-25T20:57:45.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:45 smithi092 ceph-mon[32524]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 681 B/s wr, 6 op/s 2024-04-25T20:57:45.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:45 smithi186 ceph-mon[32529]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 681 B/s wr, 6 op/s 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T20:57:46.667 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:46 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T20:57:46.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T20:57:46.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T20:57:46.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-25T20:57:46.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T20:57:46.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-25T20:57:46.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T20:57:46.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-04-25T20:57:46.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T20:57:46.784 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:46 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 -- 172.21.15.47:0/1288732534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0103450 msgr2=0x7f10f010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/1288732534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0103450 0x7f10f010f8b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f10e800b0a0 tx=0x7f10e800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 -- 172.21.15.47:0/1288732534 shutdown_connections 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/1288732534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0103450 0x7f10f010f8b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/1288732534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f0108a50 0x7f10f0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/1288732534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f0102a90 0x7f10f0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 -- 172.21.15.47:0/1288732534 >> 172.21.15.47:0/1288732534 conn(0x7f10f0079ba0 msgr2=0x7f10f0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 -- 172.21.15.47:0/1288732534 shutdown_connections 2024-04-25T20:57:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.900+0000 7f10f4cd1700 1 -- 172.21.15.47:0/1288732534 wait complete. 2024-04-25T20:57:46.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10f4cd1700 1 Processor -- start 2024-04-25T20:57:46.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10f4cd1700 1 -- start start 2024-04-25T20:57:46.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10f4cd1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f0102a90 0x7f10f0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10f4cd1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f0103450 0x7f10f0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10f4cd1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0108a50 0x7f10f019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10f4cd1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f10f0112c60 con 0x7f10f0108a50 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10f4cd1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10f0112ae0 con 0x7f10f0103450 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10f4cd1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f10f0112de0 con 0x7f10f0102a90 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10eed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0108a50 0x7f10f019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10eed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0108a50 0x7f10f019da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45042/0 (socket says 172.21.15.47:45042) 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10eed9d700 1 -- 172.21.15.47:0/2287243974 learned_addr learned my addr 172.21.15.47:0/2287243974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10ee59c700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f0102a90 0x7f10f0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10edd9b700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f0103450 0x7f10f0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10eed9d700 1 -- 172.21.15.47:0/2287243974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f0102a90 msgr2=0x7f10f0199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10eed9d700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f0102a90 0x7f10f0199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10eed9d700 1 -- 172.21.15.47:0/2287243974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f0103450 msgr2=0x7f10f0199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10eed9d700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f0103450 0x7f10f0199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.901+0000 7f10eed9d700 1 -- 172.21.15.47:0/2287243974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f019e140 con 0x7f10f0108a50 2024-04-25T20:57:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.902+0000 7f10eed9d700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0108a50 0x7f10f019da00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10e800b830 tx=0x7f10e8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.902+0000 7f10df7fe700 1 -- 172.21.15.47:0/2287243974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8021070 con 0x7f10f0108a50 2024-04-25T20:57:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.902+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f019e3d0 con 0x7f10f0108a50 2024-04-25T20:57:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.902+0000 7f10df7fe700 1 -- 172.21.15.47:0/2287243974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10e8007e30 con 0x7f10f0108a50 2024-04-25T20:57:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.902+0000 7f10df7fe700 1 -- 172.21.15.47:0/2287243974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e80047d0 con 0x7f10f0108a50 2024-04-25T20:57:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.902+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f01a5cb0 con 0x7f10f0108a50 2024-04-25T20:57:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.903+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f0069730 con 0x7f10f0108a50 2024-04-25T20:57:46.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.904+0000 7f10df7fe700 1 -- 172.21.15.47:0/2287243974 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f10e8007980 con 0x7f10f0108a50 2024-04-25T20:57:46.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.905+0000 7f10df7fe700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f10d80a3cd0 0x7f10d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.905+0000 7f10ee59c700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f10d80a3cd0 0x7f10d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.905+0000 7f10df7fe700 1 -- 172.21.15.47:0/2287243974 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f10e800f020 con 0x7f10f0108a50 2024-04-25T20:57:46.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.905+0000 7f10ee59c700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f10d80a3cd0 0x7f10d80a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009a10 tx=0x7f10e4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:46.908+0000 7f10df7fe700 1 -- 172.21.15.47:0/2287243974 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10e802a020 con 0x7f10f0108a50 2024-04-25T20:57:46.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T20:57:46.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T20:57:46.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-25T20:57:46.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T20:57:46.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-25T20:57:46.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T20:57:46.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-04-25T20:57:46.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T20:57:46.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:46 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T20:57:47.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.149+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10f002df80 con 0x7f10f0108a50 2024-04-25T20:57:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.151+0000 7f10df7fe700 1 -- 172.21.15.47:0/2287243974 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22599 (secure 0 0 0) 0x7f10e809e820 con 0x7f10f0108a50 2024-04-25T20:57:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:57:47.152 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":90,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:57:46.514798+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f10d80a3cd0 msgr2=0x7f10d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f10d80a3cd0 0x7f10d80a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009a10 tx=0x7f10e4009190 comp rx=0 tx=0).stop 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0108a50 msgr2=0x7f10f019da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0108a50 0x7f10f019da00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10e800b830 tx=0x7f10e8004ce0 comp rx=0 tx=0).stop 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 shutdown_connections 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f10d80a3cd0 0x7f10d80a6190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10f0108a50 0x7f10f019da00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f0102a90 0x7f10f0199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 --2- 172.21.15.47:0/2287243974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f0103450 0x7f10f0199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:47.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 >> 172.21.15.47:0/2287243974 conn(0x7f10f0079ba0 msgr2=0x7f10f0101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:47.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 shutdown_connections 2024-04-25T20:57:47.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:47.153+0000 7f10f4cd1700 1 -- 172.21.15.47:0/2287243974 wait complete. 2024-04-25T20:57:47.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 681 B/s wr, 7 op/s 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:47.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:47 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2287243974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:47.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 681 B/s wr, 7 op/s 2024-04-25T20:57:47.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T20:57:47.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T20:57:47.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:47.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:47 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2287243974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:47.796 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T20:57:47.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:57:47.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 681 B/s wr, 7 op/s 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:47.902 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:47 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2287243974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:48.715 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:48 smithi092 ceph-mon[32524]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T20:57:48.838 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:48 smithi047 ceph-mon[29195]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T20:57:48.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:48 smithi186 ceph-mon[32529]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T20:57:49.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:49 smithi047 ceph-mon[29195]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 638 B/s wr, 6 op/s 2024-04-25T20:57:49.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:49.837 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:49.837 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:49.837 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:49 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:49.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:49 smithi186 ceph-mon[32529]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 638 B/s wr, 6 op/s 2024-04-25T20:57:49.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:49.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:49.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:49.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:49 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:50.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:49 smithi092 ceph-mon[32524]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 638 B/s wr, 6 op/s 2024-04-25T20:57:50.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:50.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:50.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:50.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:49 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2242797079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c103480 msgr2=0x7f479c10f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2242797079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c103480 0x7f479c10f8d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4794009990 tx=0x7f4794009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2242797079 shutdown_connections 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2242797079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c103480 0x7f479c10f8d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2242797079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479c108a70 0x7f479c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2242797079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c102ac0 0x7f479c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2242797079 >> 172.21.15.47:0/2242797079 conn(0x7f479c079bb0 msgr2=0x7f479c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2242797079 shutdown_connections 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2242797079 wait complete. 2024-04-25T20:57:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.367+0000 7f47a1f6e700 1 Processor -- start 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f47a1f6e700 1 -- start start 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f47a1f6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c102ac0 0x7f479c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f47a1f6e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479c103480 0x7f479c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f47a1f6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c108a70 0x7f479c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f47a1f6e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f479c112c00 con 0x7f479c102ac0 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f47a1f6e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f479c112a80 con 0x7f479c108a70 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f47a1f6e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f479c112d80 con 0x7f479c103480 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c102ac0 0x7f479c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c102ac0 0x7f479c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45058/0 (socket says 172.21.15.47:45058) 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479b7fe700 1 -- 172.21.15.47:0/2494934820 learned_addr learned my addr 172.21.15.47:0/2494934820 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479affd700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479c103480 0x7f479c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479bfff700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c108a70 0x7f479c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479b7fe700 1 -- 172.21.15.47:0/2494934820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479c103480 msgr2=0x7f479c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479b7fe700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479c103480 0x7f479c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479b7fe700 1 -- 172.21.15.47:0/2494934820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c108a70 msgr2=0x7f479c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479b7fe700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c108a70 0x7f479c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479b7fe700 1 -- 172.21.15.47:0/2494934820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c1a7590 con 0x7f479c102ac0 2024-04-25T20:57:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.368+0000 7f479bfff700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c108a70 0x7f479c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.369+0000 7f479b7fe700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c102ac0 0x7f479c074d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f478c00ef10 tx=0x7f478c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.369+0000 7f4798ff9700 1 -- 172.21.15.47:0/2494934820 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c00cad0 con 0x7f479c102ac0 2024-04-25T20:57:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.369+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c1a7820 con 0x7f479c102ac0 2024-04-25T20:57:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.369+0000 7f4798ff9700 1 -- 172.21.15.47:0/2494934820 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c019070 con 0x7f479c102ac0 2024-04-25T20:57:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.369+0000 7f4798ff9700 1 -- 172.21.15.47:0/2494934820 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c0146e0 con 0x7f479c102ac0 2024-04-25T20:57:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.369+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c1a7d60 con 0x7f479c102ac0 2024-04-25T20:57:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.370+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c069730 con 0x7f479c102ac0 2024-04-25T20:57:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.371+0000 7f4798ff9700 1 -- 172.21.15.47:0/2494934820 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f478c012050 con 0x7f479c102ac0 2024-04-25T20:57:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.372+0000 7f4798ff9700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47880a3cd0 0x7f47880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.372+0000 7f479affd700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47880a3cd0 0x7f47880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.372+0000 7f4798ff9700 1 -- 172.21.15.47:0/2494934820 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f478c0d05f0 con 0x7f479c102ac0 2024-04-25T20:57:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.372+0000 7f479affd700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47880a3cd0 0x7f47880a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4784005950 tx=0x7f47840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.376+0000 7f4798ff9700 1 -- 172.21.15.47:0/2494934820 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f478c010040 con 0x7f479c102ac0 2024-04-25T20:57:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.618+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f479c02df80 con 0x7f479c102ac0 2024-04-25T20:57:50.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.620+0000 7f4798ff9700 1 -- 172.21.15.47:0/2494934820 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22599 (secure 0 0 0) 0x7f478c09ce60 con 0x7f479c102ac0 2024-04-25T20:57:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:57:50.621 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:57:47.520070+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:57:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.622+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47880a3cd0 msgr2=0x7f47880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.622+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47880a3cd0 0x7f47880a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4784005950 tx=0x7f47840058e0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c102ac0 msgr2=0x7f479c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c102ac0 0x7f479c074d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f478c00ef10 tx=0x7f478c00c3a0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 shutdown_connections 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f47880a3cd0 0x7f47880a6190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f479c102ac0 0x7f479c074d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c108a70 0x7f479c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 --2- 172.21.15.47:0/2494934820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479c103480 0x7f479c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 >> 172.21.15.47:0/2494934820 conn(0x7f479c079bb0 msgr2=0x7f479c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 shutdown_connections 2024-04-25T20:57:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:50.623+0000 7f47a1f6e700 1 -- 172.21.15.47:0/2494934820 wait complete. 2024-04-25T20:57:51.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T20:57:51.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T20:57:51.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T20:57:51.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T20:57:51.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T20:57:51.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T20:57:51.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T20:57:51.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T20:57:51.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T20:57:51.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T20:57:51.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T20:57:51.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T20:57:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:51 smithi047 ceph-mon[29195]: pgmap v323: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:51 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:51.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:51 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2494934820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:51.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:51 smithi186 ceph-mon[32529]: pgmap v323: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:51.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:51 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:51.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:51 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:51.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:51 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2494934820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:52.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:51 smithi092 ceph-mon[32524]: pgmap v323: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:52.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:51 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:52.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:51 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:52.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:51 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2494934820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:52.099 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:52.116 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:52.119 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:52.851 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:52.851 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:52.851 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:52.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:52.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:52.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:53.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:53.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:53.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:57:53.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:53 smithi047 ceph-mon[29195]: pgmap v324: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:53.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:53 smithi186 ceph-mon[32529]: pgmap v324: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:54.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:53 smithi092 ceph-mon[32524]: pgmap v324: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:55.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:55 smithi047 ceph-mon[29195]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:55.849 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:55 smithi092 ceph-mon[32524]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:55 smithi186 ceph-mon[32529]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 -- 172.21.15.47:0/1889962728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 msgr2=0x7fa18006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 --2- 172.21.15.47:0/1889962728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 0x7fa18006abb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa174009990 tx=0x7fa174009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 -- 172.21.15.47:0/1889962728 shutdown_connections 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 --2- 172.21.15.47:0/1889962728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18006b0f0 0x7fa18010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 --2- 172.21.15.47:0/1889962728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180108a30 0x7fa180108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 --2- 172.21.15.47:0/1889962728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 0x7fa18006abb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 -- 172.21.15.47:0/1889962728 >> 172.21.15.47:0/1889962728 conn(0x7fa1800788f0 msgr2=0x7fa180078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 -- 172.21.15.47:0/1889962728 shutdown_connections 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.014+0000 7fa1852e8700 1 -- 172.21.15.47:0/1889962728 wait complete. 2024-04-25T20:57:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa1852e8700 1 Processor -- start 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa1852e8700 1 -- start start 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa1852e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 0x7fa180101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa1852e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18006b0f0 0x7fa180101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa1852e8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180108a30 0x7fa180102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa1852e8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa180112c20 con 0x7fa18006a750 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa1852e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa180112aa0 con 0x7fa18006b0f0 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa1852e8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa180112da0 con 0x7fa180108a30 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 0x7fa180101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 0x7fa180101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46536/0 (socket says 172.21.15.47:46536) 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17effd700 1 -- 172.21.15.47:0/3780060813 learned_addr learned my addr 172.21.15.47:0/3780060813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17e7fc700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18006b0f0 0x7fa180101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17f7fe700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180108a30 0x7fa180102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17effd700 1 -- 172.21.15.47:0/3780060813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180108a30 msgr2=0x7fa180102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17effd700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa180108a30 0x7fa180102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17effd700 1 -- 172.21.15.47:0/3780060813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18006b0f0 msgr2=0x7fa180101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17effd700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18006b0f0 0x7fa180101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.015+0000 7fa17effd700 1 -- 172.21.15.47:0/3780060813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1801a7690 con 0x7fa18006a750 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.016+0000 7fa17effd700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 0x7fa180101a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa17000d980 tx=0x7fa17000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.016+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170017070 con 0x7fa18006a750 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.016+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1801a7980 con 0x7fa18006a750 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.016+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa170005020 con 0x7fa18006a750 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.016+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17000fa10 con 0x7fa18006a750 2024-04-25T20:57:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.016+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1801a7ec0 con 0x7fa18006a750 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.017+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa180063110 con 0x7fa18006a750 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.018+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa170004a30 con 0x7fa18006a750 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.019+0000 7fa167fff700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1680a3cd0 0x7fa1680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.019+0000 7fa17e7fc700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1680a3cd0 0x7fa1680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.019+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fa1700d08d0 con 0x7fa18006a750 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.019+0000 7fa167fff700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] conn(0x7fa1680ad8b0 0x7fa1680afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.019+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 --> [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1680b0400 con 0x7fa1680ad8b0 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.019+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa1700d0d20 con 0x7fa18006a750 2024-04-25T20:57:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.020+0000 7fa17f7fe700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] conn(0x7fa1680ad8b0 0x7fa1680afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.020+0000 7fa17e7fc700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1680a3cd0 0x7fa1680a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa180078d80 tx=0x7fa174005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.020+0000 7fa17f7fe700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] conn(0x7fa1680ad8b0 0x7fa1680afd30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.021+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 <== osd.6 v2:172.21.15.47:6818/2069873922 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1680b0400 con 0x7fa1680ad8b0 2024-04-25T20:57:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.037+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 --> [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa180063590 con 0x7fa1680ad8b0 2024-04-25T20:57:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.037+0000 7fa167fff700 1 -- 172.21.15.47:0/3780060813 <== osd.6 v2:172.21.15.47:6818/2069873922 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa180063590 con 0x7fa1680ad8b0 2024-04-25T20:57:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] conn(0x7fa1680ad8b0 msgr2=0x7fa1680afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] conn(0x7fa1680ad8b0 0x7fa1680afd30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1680a3cd0 msgr2=0x7fa1680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1680a3cd0 0x7fa1680a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa180078d80 tx=0x7fa174005fd0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 msgr2=0x7fa180101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 --2- 172.21.15.47:0/3780060813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa18006a750 0x7fa180101a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa17000d980 tx=0x7fa17000dc90 comp rx=0 tx=0).stop 2024-04-25T20:57:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa17f7fe700 1 -- 172.21.15.47:0/3780060813 reap_dead start 2024-04-25T20:57:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 shutdown_connections 2024-04-25T20:57:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 >> 172.21.15.47:0/3780060813 conn(0x7fa1800788f0 msgr2=0x7fa1800ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 shutdown_connections 2024-04-25T20:57:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:56.038+0000 7fa1852e8700 1 -- 172.21.15.47:0/3780060813 wait complete. 2024-04-25T20:57:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.471+0000 7feb31361700 1 -- 172.21.15.47:0/3676336531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb2c06b680 msgr2=0x7feb2c06bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.471+0000 7feb31361700 1 --2- 172.21.15.47:0/3676336531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb2c06b680 0x7feb2c06bae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb20009990 tx=0x7feb20009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.471+0000 7feb31361700 1 -- 172.21.15.47:0/3676336531 shutdown_connections 2024-04-25T20:57:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.471+0000 7feb31361700 1 --2- 172.21.15.47:0/3676336531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb2c1054b0 0x7feb2c105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.471+0000 7feb31361700 1 --2- 172.21.15.47:0/3676336531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c1079a0 0x7feb2c06b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.471+0000 7feb31361700 1 --2- 172.21.15.47:0/3676336531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb2c06b680 0x7feb2c06bae0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.471+0000 7feb31361700 1 -- 172.21.15.47:0/3676336531 >> 172.21.15.47:0/3676336531 conn(0x7feb2c078980 msgr2=0x7feb2c078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.471+0000 7feb31361700 1 -- 172.21.15.47:0/3676336531 shutdown_connections 2024-04-25T20:57:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.472+0000 7feb31361700 1 -- 172.21.15.47:0/3676336531 wait complete. 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.472+0000 7feb31361700 1 Processor -- start 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb31361700 1 -- start start 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb31361700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06b680 0x7feb2c1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb31361700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb2c1054b0 0x7feb2c1a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb31361700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb2c1079a0 0x7feb2c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb31361700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c119290 con 0x7feb2c1079a0 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb31361700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c119110 con 0x7feb2c1054b0 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb31361700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c119410 con 0x7feb2c06b680 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb2c1054b0 0x7feb2c1a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb2c1054b0 0x7feb2c1a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:33968/0 (socket says 172.21.15.47:33968) 2024-04-25T20:57:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb2c1079a0 0x7feb2c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2a7fc700 1 -- 172.21.15.47:0/1485988295 learned_addr learned my addr 172.21.15.47:0/1485988295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2affd700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06b680 0x7feb2c1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2a7fc700 1 -- 172.21.15.47:0/1485988295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06b680 msgr2=0x7feb2c1a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2a7fc700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06b680 0x7feb2c1a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2a7fc700 1 -- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb2c1079a0 msgr2=0x7feb2c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2a7fc700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb2c1079a0 0x7feb2c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.473+0000 7feb2a7fc700 1 -- 172.21.15.47:0/1485988295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c19fd10 con 0x7feb2c1054b0 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.474+0000 7feb2affd700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06b680 0x7feb2c1a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.474+0000 7feb2a7fc700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb2c1054b0 0x7feb2c1a5930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c010 tx=0x7feb200057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.474+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20026070 con 0x7feb2c1054b0 2024-04-25T20:57:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.474+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c19ffa0 con 0x7feb2c1054b0 2024-04-25T20:57:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.474+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb20003f20 con 0x7feb2c1054b0 2024-04-25T20:57:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.474+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2000fc40 con 0x7feb2c1054b0 2024-04-25T20:57:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.474+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c1ac2c0 con 0x7feb2c1054b0 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.475+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb2c104eb0 con 0x7feb2c1054b0 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.476+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feb20029070 con 0x7feb2c1054b0 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.477+0000 7feb13fff700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7feb140a3d10 0x7feb140a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.477+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7feb200d1a90 con 0x7feb2c1054b0 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.477+0000 7feb2affd700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7feb140a3d10 0x7feb140a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.477+0000 7feb13fff700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] conn(0x7feb140ad8f0 0x7feb140afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.477+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 --> [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb140b0440 con 0x7feb140ad8f0 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.477+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7feb200d1e40 con 0x7feb2c1054b0 2024-04-25T20:57:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.477+0000 7feb2b7fe700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] conn(0x7feb140ad8f0 0x7feb140afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.478+0000 7feb2b7fe700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] conn(0x7feb140ad8f0 0x7feb140afd70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.478+0000 7feb2affd700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7feb140a3d10 0x7feb140a61d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feb1c005e10 tx=0x7feb1c005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.478+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 <== osd.0 v2:172.21.15.47:6802/3737942264 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb140b0440 con 0x7feb140ad8f0 2024-04-25T20:57:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.495+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 --> [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb2c101f90 con 0x7feb140ad8f0 2024-04-25T20:57:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb13fff700 1 -- 172.21.15.47:0/1485988295 <== osd.0 v2:172.21.15.47:6802/3737942264 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feb2c101f90 con 0x7feb140ad8f0 2024-04-25T20:57:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] conn(0x7feb140ad8f0 msgr2=0x7feb140afd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb31361700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] conn(0x7feb140ad8f0 0x7feb140afd70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7feb140a3d10 msgr2=0x7feb140a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb31361700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7feb140a3d10 0x7feb140a61d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feb1c005e10 tx=0x7feb1c005d60 comp rx=0 tx=0).stop 2024-04-25T20:57:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb2c1054b0 msgr2=0x7feb2c1a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb31361700 1 --2- 172.21.15.47:0/1485988295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb2c1054b0 0x7feb2c1a5930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c010 tx=0x7feb200057c0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb2b7fe700 1 -- 172.21.15.47:0/1485988295 reap_dead start 2024-04-25T20:57:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 shutdown_connections 2024-04-25T20:57:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.496+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 >> 172.21.15.47:0/1485988295 conn(0x7feb2c078980 msgr2=0x7feb2c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.497+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 shutdown_connections 2024-04-25T20:57:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:57.497+0000 7feb31361700 1 -- 172.21.15.47:0/1485988295 wait complete. 2024-04-25T20:57:57.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:57 smithi047 ceph-mon[29195]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:57:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:57 smithi186 ceph-mon[32529]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:57:58.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:57 smithi092 ceph-mon[32524]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:57:58.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 -- 172.21.15.47:0/4289749449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8102300 msgr2=0x7fa7f810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:58.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 --2- 172.21.15.47:0/4289749449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8102300 0x7fa7f810e7f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000b0a0 tx=0x7fa7f000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 -- 172.21.15.47:0/4289749449 shutdown_connections 2024-04-25T20:57:58.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 --2- 172.21.15.47:0/4289749449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8102300 0x7fa7f810e7f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 --2- 172.21.15.47:0/4289749449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7f8107990 0x7fa7f8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 --2- 172.21.15.47:0/4289749449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f8101940 0x7fa7f8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 -- 172.21.15.47:0/4289749449 >> 172.21.15.47:0/4289749449 conn(0x7fa7f80788f0 msgr2=0x7fa7f8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 -- 172.21.15.47:0/4289749449 shutdown_connections 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.217+0000 7fa7fce69700 1 -- 172.21.15.47:0/4289749449 wait complete. 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7fce69700 1 Processor -- start 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7fce69700 1 -- start start 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7fce69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f8101940 0x7fa7f819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7fce69700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7f8102300 0x7fa7f819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7fce69700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8107990 0x7fa7f819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7fce69700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8111b60 con 0x7fa7f8107990 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7fce69700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81119e0 con 0x7fa7f8101940 2024-04-25T20:57:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7fce69700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8111ce0 con 0x7fa7f8102300 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7f6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8107990 0x7fa7f819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7f6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8107990 0x7fa7f819fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46582/0 (socket says 172.21.15.47:46582) 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.218+0000 7fa7f6d9d700 1 -- 172.21.15.47:0/603003307 learned_addr learned my addr 172.21.15.47:0/603003307 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7f659c700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f8101940 0x7fa7f819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7edd9b700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7f8102300 0x7fa7f819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7f6d9d700 1 -- 172.21.15.47:0/603003307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7f8102300 msgr2=0x7fa7f819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7f6d9d700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7f8102300 0x7fa7f819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7f6d9d700 1 -- 172.21.15.47:0/603003307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f8101940 msgr2=0x7fa7f819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7f6d9d700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f8101940 0x7fa7f819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7f6d9d700 1 -- 172.21.15.47:0/603003307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f81a0350 con 0x7fa7f8107990 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7f659c700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f8101940 0x7fa7f819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7f6d9d700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8107990 0x7fa7f819fc10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00062a0 tx=0x7fa7f0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0021070 con 0x7fa7f8107990 2024-04-25T20:57:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f81a05e0 con 0x7fa7f8107990 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f0007d00 con 0x7fa7f8107990 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f001d450 con 0x7fa7f8107990 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.219+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f81a7ec0 con 0x7fa7f8107990 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.220+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7f806b080 con 0x7fa7f8107990 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.221+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa7f0004000 con 0x7fa7f8107990 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.222+0000 7fa7effff700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.222+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fa7f000f020 con 0x7fa7f8107990 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.222+0000 7fa7f659c700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.222+0000 7fa7effff700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] conn(0x7fa7e40ad8b0 0x7fa7e40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.222+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 --> [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7f00020b0 con 0x7fa7e40ad8b0 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.222+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa7f00d22e0 con 0x7fa7f8107990 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.222+0000 7fa7edd9b700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] conn(0x7fa7e40ad8b0 0x7fa7e40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.223+0000 7fa7f659c700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80099d0 tx=0x7fa7e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.223+0000 7fa7edd9b700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] conn(0x7fa7e40ad8b0 0x7fa7e40afd30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.224+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 <== osd.3 v2:172.21.15.47:6810/2086583036 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7f00020b0 con 0x7fa7e40ad8b0 2024-04-25T20:57:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.240+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 --> [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa7f8103060 con 0x7fa7e40ad8b0 2024-04-25T20:57:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.240+0000 7fa7effff700 1 -- 172.21.15.47:0/603003307 <== osd.3 v2:172.21.15.47:6810/2086583036 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa7f8103060 con 0x7fa7e40ad8b0 2024-04-25T20:57:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] conn(0x7fa7e40ad8b0 msgr2=0x7fa7e40afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] conn(0x7fa7e40ad8b0 0x7fa7e40afd30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa7e40a3cd0 msgr2=0x7fa7e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80099d0 tx=0x7fa7e8008040 comp rx=0 tx=0).stop 2024-04-25T20:57:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8107990 msgr2=0x7fa7f819fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 --2- 172.21.15.47:0/603003307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7f8107990 0x7fa7f819fc10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00062a0 tx=0x7fa7f0004ed0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7f6d9d700 1 -- 172.21.15.47:0/603003307 reap_dead start 2024-04-25T20:57:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 shutdown_connections 2024-04-25T20:57:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 >> 172.21.15.47:0/603003307 conn(0x7fa7f80788f0 msgr2=0x7fa7f806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 shutdown_connections 2024-04-25T20:57:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:57:58.241+0000 7fa7fce69700 1 -- 172.21.15.47:0/603003307 wait complete. 2024-04-25T20:57:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:59.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:57:59 smithi047 ceph-mon[29195]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:57:59.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:57:59 smithi186 ceph-mon[32529]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:58:00.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:57:59 smithi092 ceph-mon[32524]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:58:00.626 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:01.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:01 smithi047 ceph-mon[29195]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:58:01.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:01 smithi186 ceph-mon[32529]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:58:02.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:01 smithi092 ceph-mon[32524]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T20:58:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 -- 172.21.15.47:0/2168162838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14107990 msgr2=0x7f2a14107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 --2- 172.21.15.47:0/2168162838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14107990 0x7f2a14107d70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009990 tx=0x7f2a10009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 -- 172.21.15.47:0/2168162838 shutdown_connections 2024-04-25T20:58:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 --2- 172.21.15.47:0/2168162838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14107990 0x7f2a14107d70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 --2- 172.21.15.47:0/2168162838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a14102300 0x7f2a1410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 --2- 172.21.15.47:0/2168162838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a14101940 0x7f2a14101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 -- 172.21.15.47:0/2168162838 >> 172.21.15.47:0/2168162838 conn(0x7f2a140788f0 msgr2=0x7f2a14078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 -- 172.21.15.47:0/2168162838 shutdown_connections 2024-04-25T20:58:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.220+0000 7f2a1bae3700 1 -- 172.21.15.47:0/2168162838 wait complete. 2024-04-25T20:58:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1bae3700 1 Processor -- start 2024-04-25T20:58:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1bae3700 1 -- start start 2024-04-25T20:58:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1bae3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14101940 0x7f2a141980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1bae3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a14102300 0x7f2a141985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1bae3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a14107990 0x7f2a1419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1bae3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14111b60 con 0x7f2a14101940 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1bae3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a141119e0 con 0x7f2a14102300 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1bae3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14111ce0 con 0x7f2a14107990 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1987f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14101940 0x7f2a141980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1987f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14101940 0x7f2a141980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46602/0 (socket says 172.21.15.47:46602) 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.221+0000 7f2a1987f700 1 -- 172.21.15.47:0/1289669118 learned_addr learned my addr 172.21.15.47:0/1289669118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1907e700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a14102300 0x7f2a141985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1a080700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a14107990 0x7f2a1419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1987f700 1 -- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a14107990 msgr2=0x7f2a1419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1987f700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a14107990 0x7f2a1419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1987f700 1 -- 172.21.15.47:0/1289669118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a14102300 msgr2=0x7f2a141985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1987f700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a14102300 0x7f2a141985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1987f700 1 -- 172.21.15.47:0/1289669118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1419d0c0 con 0x7f2a14101940 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1907e700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a14102300 0x7f2a141985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1a080700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a14107990 0x7f2a1419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1987f700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14101940 0x7f2a141980b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a10000c00 tx=0x7f2a100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10027070 con 0x7f2a14101940 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1419d350 con 0x7f2a14101940 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a100041d0 con 0x7f2a14101940 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1000f460 con 0x7f2a14101940 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.222+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a141a4c30 con 0x7f2a14101940 2024-04-25T20:58:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.223+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a14063190 con 0x7f2a14101940 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.224+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2a1002a070 con 0x7f2a14101940 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.225+0000 7f2a0affd700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a000a3cd0 0x7f2a000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.225+0000 7f2a1907e700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a000a3cd0 0x7f2a000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.225+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f2a100d1c70 con 0x7f2a14101940 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.225+0000 7f2a0affd700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] conn(0x7f2a000ad8b0 0x7f2a000afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.225+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 --> [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a10019e80 con 0x7f2a000ad8b0 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.225+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f2a100d20c0 con 0x7f2a14101940 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.226+0000 7f2a1907e700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a000a3cd0 0x7f2a000a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a140790a0 tx=0x7f2a04008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.226+0000 7f2a1a080700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] conn(0x7f2a000ad8b0 0x7f2a000afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.226+0000 7f2a1a080700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] conn(0x7f2a000ad8b0 0x7f2a000afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.227+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 <== osd.11 v2:172.21.15.186:6824/936644453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a10019e80 con 0x7f2a000ad8b0 2024-04-25T20:58:03.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.243+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 --> [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2a140635c0 con 0x7f2a000ad8b0 2024-04-25T20:58:03.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.244+0000 7f2a0affd700 1 -- 172.21.15.47:0/1289669118 <== osd.11 v2:172.21.15.186:6824/936644453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2a140635c0 con 0x7f2a000ad8b0 2024-04-25T20:58:03.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.244+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] conn(0x7f2a000ad8b0 msgr2=0x7f2a000afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.244+0000 7f2a1bae3700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] conn(0x7f2a000ad8b0 0x7f2a000afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.244+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a000a3cd0 msgr2=0x7f2a000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.244+0000 7f2a1bae3700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f2a000a3cd0 0x7f2a000a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a140790a0 tx=0x7f2a04008040 comp rx=0 tx=0).stop 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.244+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14101940 msgr2=0x7f2a141980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.244+0000 7f2a1bae3700 1 --2- 172.21.15.47:0/1289669118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a14101940 0x7f2a141980b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a10000c00 tx=0x7f2a100047b0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.245+0000 7f2a1a080700 1 -- 172.21.15.47:0/1289669118 reap_dead start 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.245+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 shutdown_connections 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.245+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 >> 172.21.15.47:0/1289669118 conn(0x7f2a140788f0 msgr2=0x7f2a14100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.245+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 shutdown_connections 2024-04-25T20:58:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.245+0000 7f2a1bae3700 1 -- 172.21.15.47:0/1289669118 wait complete. 2024-04-25T20:58:03.538 INFO:teuthology.orchestra.run.smithi047.stdout:201863462966 2024-04-25T20:58:03.539 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T20:58:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 -- 172.21.15.47:0/651407393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf28108a20 msgr2=0x7faf28108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 --2- 172.21.15.47:0/651407393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf28108a20 0x7faf28108e00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faf18009990 tx=0x7faf18009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 -- 172.21.15.47:0/651407393 shutdown_connections 2024-04-25T20:58:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 --2- 172.21.15.47:0/651407393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf28108a20 0x7faf28108e00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 --2- 172.21.15.47:0/651407393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf2806b980 0x7faf2810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 --2- 172.21.15.47:0/651407393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf2806afe0 0x7faf2806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 -- 172.21.15.47:0/651407393 >> 172.21.15.47:0/651407393 conn(0x7faf280788b0 msgr2=0x7faf28078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:03.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 -- 172.21.15.47:0/651407393 shutdown_connections 2024-04-25T20:58:03.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.732+0000 7faf2e172700 1 -- 172.21.15.47:0/651407393 wait complete. 2024-04-25T20:58:03.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.733+0000 7faf2e172700 1 Processor -- start 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.733+0000 7faf2e172700 1 -- start start 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf2e172700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf2806afe0 0x7faf28101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf2e172700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf2806b980 0x7faf28101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf2e172700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf28108a20 0x7faf28102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf2e172700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faf28112bb0 con 0x7faf2806afe0 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf2e172700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faf28112a30 con 0x7faf2806b980 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf2e172700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faf28112d30 con 0x7faf28108a20 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf2806afe0 0x7faf28101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf2806afe0 0x7faf28101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53736/0 (socket says 172.21.15.47:53736) 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 -- 172.21.15.47:0/3246863021 learned_addr learned my addr 172.21.15.47:0/3246863021 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf1effd700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf2806b980 0x7faf28101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf27fff700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf28108a20 0x7faf28102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 -- 172.21.15.47:0/3246863021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf28108a20 msgr2=0x7faf28102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf28108a20 0x7faf28102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 -- 172.21.15.47:0/3246863021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf2806b980 msgr2=0x7faf28101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf2806b980 0x7faf28101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 -- 172.21.15.47:0/3246863021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf281a7650 con 0x7faf2806afe0 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf27fff700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf28108a20 0x7faf28102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.734+0000 7faf277fe700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf2806afe0 0x7faf28101a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faf18000c00 tx=0x7faf18009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.735+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18027070 con 0x7faf2806afe0 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.735+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf18022470 con 0x7faf2806afe0 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.735+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18017650 con 0x7faf2806afe0 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.735+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf281a78e0 con 0x7faf2806afe0 2024-04-25T20:58:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.735+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf281a7ce0 con 0x7faf2806afe0 2024-04-25T20:58:03.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.736+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf280630d0 con 0x7faf2806afe0 2024-04-25T20:58:03.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.737+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faf1802a070 con 0x7faf2806afe0 2024-04-25T20:58:03.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.738+0000 7faf257fa700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7faf140a3d50 0x7faf140a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.738+0000 7faf1effd700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7faf140a3d50 0x7faf140a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.738+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7faf180d12b0 con 0x7faf2806afe0 2024-04-25T20:58:03.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.738+0000 7faf257fa700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] conn(0x7faf140ad930 0x7faf140afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.738+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 --> [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf140b0480 con 0x7faf140ad930 2024-04-25T20:58:03.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.738+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7faf180d8050 con 0x7faf2806afe0 2024-04-25T20:58:03.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.738+0000 7faf27fff700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] conn(0x7faf140ad930 0x7faf140afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.738+0000 7faf1effd700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7faf140a3d50 0x7faf140a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faf28078f90 tx=0x7faf1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:03.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.739+0000 7faf27fff700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] conn(0x7faf140ad930 0x7faf140afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:03.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.740+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 <== osd.1 v2:172.21.15.92:6800/3070209756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf140b0480 con 0x7faf140ad930 2024-04-25T20:58:03.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.764+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 --> [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf280635a0 con 0x7faf140ad930 2024-04-25T20:58:03.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:03 smithi047 ceph-mon[29195]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:03.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf257fa700 1 -- 172.21.15.47:0/3246863021 <== osd.1 v2:172.21.15.92:6800/3070209756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf280635a0 con 0x7faf140ad930 2024-04-25T20:58:03.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 >> [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] conn(0x7faf140ad930 msgr2=0x7faf140afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] conn(0x7faf140ad930 0x7faf140afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7faf140a3d50 msgr2=0x7faf140a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7faf140a3d50 0x7faf140a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faf28078f90 tx=0x7faf1000b410 comp rx=0 tx=0).stop 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf2806afe0 msgr2=0x7faf28101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 --2- 172.21.15.47:0/3246863021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf2806afe0 0x7faf28101a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faf18000c00 tx=0x7faf18009580 comp rx=0 tx=0).stop 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf27fff700 1 -- 172.21.15.47:0/3246863021 reap_dead start 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 shutdown_connections 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 >> 172.21.15.47:0/3246863021 conn(0x7faf280788b0 msgr2=0x7faf280ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 shutdown_connections 2024-04-25T20:58:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:03.765+0000 7faf2e172700 1 -- 172.21.15.47:0/3246863021 wait complete. 2024-04-25T20:58:03.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:03 smithi186 ceph-mon[32529]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:04.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:03 smithi092 ceph-mon[32524]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:05 smithi047 ceph-mon[29195]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:05 smithi186 ceph-mon[32529]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:05 smithi092 ceph-mon[32524]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:07.181 INFO:teuthology.orchestra.run.smithi047.stdout:347892351006 2024-04-25T20:58:07.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T20:58:07.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:07 smithi047 ceph-mon[29195]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:07.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:07 smithi186 ceph-mon[32529]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:08.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:07 smithi092 ceph-mon[32524]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:08.042 INFO:teuthology.orchestra.run.smithi047.stdout:34359738451 2024-04-25T20:58:08.043 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T20:58:08.753 INFO:teuthology.orchestra.run.smithi047.stdout:107374182467 2024-04-25T20:58:08.754 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T20:58:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 -- 172.21.15.47:0/3993920459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808806bbb0 msgr2=0x7f808810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 --2- 172.21.15.47:0/3993920459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808806bbb0 0x7f808810f930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f808000b0a0 tx=0x7f808000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 -- 172.21.15.47:0/3993920459 shutdown_connections 2024-04-25T20:58:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 --2- 172.21.15.47:0/3993920459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808806bbb0 0x7f808810f930 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 --2- 172.21.15.47:0/3993920459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8088108a60 0x7f8088108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 --2- 172.21.15.47:0/3993920459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f808806b1f0 0x7f808806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 -- 172.21.15.47:0/3993920459 >> 172.21.15.47:0/3993920459 conn(0x7f80880789e0 msgr2=0x7f8088078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 -- 172.21.15.47:0/3993920459 shutdown_connections 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.467+0000 7f808ffb0700 1 -- 172.21.15.47:0/3993920459 wait complete. 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808ffb0700 1 Processor -- start 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808ffb0700 1 -- start start 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808ffb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f808806b1f0 0x7f808819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808ffb0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f808806bbb0 0x7f808819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808ffb0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8088108a60 0x7f808819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808ffb0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8088112c40 con 0x7f8088108a60 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808ffb0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8088112ac0 con 0x7f808806b1f0 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808ffb0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8088112dc0 con 0x7f808806bbb0 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808e54d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8088108a60 0x7f808819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808e54d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8088108a60 0x7f808819fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53758/0 (socket says 172.21.15.47:53758) 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808e54d700 1 -- 172.21.15.47:0/2478936667 learned_addr learned my addr 172.21.15.47:0/2478936667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808dd4c700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f808806b1f0 0x7f808819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808d54b700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f808806bbb0 0x7f808819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808e54d700 1 -- 172.21.15.47:0/2478936667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f808806bbb0 msgr2=0x7f808819b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808e54d700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f808806bbb0 0x7f808819b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808e54d700 1 -- 172.21.15.47:0/2478936667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f808806b1f0 msgr2=0x7f808819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808e54d700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f808806b1f0 0x7f808819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808e54d700 1 -- 172.21.15.47:0/2478936667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80881a02a0 con 0x7f8088108a60 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.468+0000 7f808d54b700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f808806bbb0 0x7f808819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.469+0000 7f808e54d700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8088108a60 0x7f808819fb60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f80800062a0 tx=0x7f8080004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.469+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080021070 con 0x7f8088108a60 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.469+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80881a0530 con 0x7f8088108a60 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.469+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8080007c80 con 0x7f8088108a60 2024-04-25T20:58:09.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.469+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808001d450 con 0x7f8088108a60 2024-04-25T20:58:09.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.469+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80881a7e10 con 0x7f8088108a60 2024-04-25T20:58:09.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.470+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8088063150 con 0x7f8088108a60 2024-04-25T20:58:09.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.472+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8080004000 con 0x7f8088108a60 2024-04-25T20:58:09.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.473+0000 7f807effd700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f80740a3d50 0x7f80740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.473+0000 7f808dd4c700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f80740a3d50 0x7f80740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.473+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f808000f020 con 0x7f8088108a60 2024-04-25T20:58:09.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.473+0000 7f807effd700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] conn(0x7f80740ad930 0x7f80740afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.473+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 --> [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f808002ee50 con 0x7f80740ad930 2024-04-25T20:58:09.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.473+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f80800d2270 con 0x7f8088108a60 2024-04-25T20:58:09.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.473+0000 7f808dd4c700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f80740a3d50 0x7f80740a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f808400abd0 tx=0x7f808400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.474+0000 7f808d54b700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] conn(0x7f80740ad930 0x7f80740afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.474+0000 7f808d54b700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] conn(0x7f80740ad930 0x7f80740afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.475+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 <== osd.7 v2:172.21.15.92:6816/1310010410 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f808002ee50 con 0x7f80740ad930 2024-04-25T20:58:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.491+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 --> [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8088063550 con 0x7f80740ad930 2024-04-25T20:58:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.492+0000 7f807effd700 1 -- 172.21.15.47:0/2478936667 <== osd.7 v2:172.21.15.92:6816/1310010410 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8088063550 con 0x7f80740ad930 2024-04-25T20:58:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.492+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 >> [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] conn(0x7f80740ad930 msgr2=0x7f80740afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.492+0000 7f808ffb0700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] conn(0x7f80740ad930 0x7f80740afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.492+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f80740a3d50 msgr2=0x7f80740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.492+0000 7f808ffb0700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f80740a3d50 0x7f80740a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f808400abd0 tx=0x7f808400a380 comp rx=0 tx=0).stop 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.492+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8088108a60 msgr2=0x7f808819fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.492+0000 7f808ffb0700 1 --2- 172.21.15.47:0/2478936667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8088108a60 0x7f808819fb60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f80800062a0 tx=0x7f8080004e20 comp rx=0 tx=0).stop 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.492+0000 7f808e54d700 1 -- 172.21.15.47:0/2478936667 reap_dead start 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.493+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 shutdown_connections 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.493+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 >> 172.21.15.47:0/2478936667 conn(0x7f80880789e0 msgr2=0x7f808810ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.493+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 shutdown_connections 2024-04-25T20:58:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:09.493+0000 7f808ffb0700 1 -- 172.21.15.47:0/2478936667 wait complete. 2024-04-25T20:58:09.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:09 smithi047 ceph-mon[29195]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:09.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:09 smithi186 ceph-mon[32529]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:10.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:09 smithi092 ceph-mon[32524]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 -- 172.21.15.47:0/465955660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 msgr2=0x7f491806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 --2- 172.21.15.47:0/465955660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 0x7f491806abb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4900009990 tx=0x7f4900009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 -- 172.21.15.47:0/465955660 shutdown_connections 2024-04-25T20:58:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 --2- 172.21.15.47:0/465955660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f491806b0f0 0x7f491810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 --2- 172.21.15.47:0/465955660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4918108a30 0x7f4918108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 --2- 172.21.15.47:0/465955660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 0x7f491806abb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 -- 172.21.15.47:0/465955660 >> 172.21.15.47:0/465955660 conn(0x7f49180788f0 msgr2=0x7f4918078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 -- 172.21.15.47:0/465955660 shutdown_connections 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.145+0000 7f491d429700 1 -- 172.21.15.47:0/465955660 wait complete. 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f491d429700 1 Processor -- start 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f491d429700 1 -- start start 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f491d429700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 0x7f4918101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f491d429700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f491806b0f0 0x7f4918101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f491d429700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4918108a30 0x7f4918102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f491d429700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4918112cb0 con 0x7f491806a750 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f491d429700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4918112b30 con 0x7f491806b0f0 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f491d429700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4918112e30 con 0x7f4918108a30 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f4916ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 0x7f4918101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f4916ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 0x7f4918101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53770/0 (socket says 172.21.15.47:53770) 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f4916ffd700 1 -- 172.21.15.47:0/1675102035 learned_addr learned my addr 172.21.15.47:0/1675102035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.146+0000 7f49167fc700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f491806b0f0 0x7f4918101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f49177fe700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4918108a30 0x7f4918102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f4916ffd700 1 -- 172.21.15.47:0/1675102035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4918108a30 msgr2=0x7f4918102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f4916ffd700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4918108a30 0x7f4918102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f4916ffd700 1 -- 172.21.15.47:0/1675102035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f491806b0f0 msgr2=0x7f4918101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f4916ffd700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f491806b0f0 0x7f4918101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f4916ffd700 1 -- 172.21.15.47:0/1675102035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49181a7800 con 0x7f491806a750 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f4916ffd700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 0x7f4918101a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f490800b530 tx=0x7f490800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908015020 con 0x7f491806a750 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4908011ce0 con 0x7f491806a750 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49181a7af0 con 0x7f491806a750 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908010e40 con 0x7f491806a750 2024-04-25T20:58:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.147+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49181a8050 con 0x7f491806a750 2024-04-25T20:58:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.148+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49180630d0 con 0x7f491806a750 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.149+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4908018020 con 0x7f491806a750 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.150+0000 7f490ffff700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f49040a3d50 0x7f49040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.150+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f49080d03a0 con 0x7f491806a750 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.150+0000 7f49167fc700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f49040a3d50 0x7f49040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.150+0000 7f490ffff700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] conn(0x7f49040ad930 0x7f49040afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.150+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 --> [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4908018ec0 con 0x7f49040ad930 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.150+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f49080d07f0 con 0x7f491806a750 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.150+0000 7f49167fc700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f49040a3d50 0x7f49040a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f490000b5c0 tx=0x7f4900005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.150+0000 7f49177fe700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] conn(0x7f49040ad930 0x7f49040afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.151+0000 7f49177fe700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] conn(0x7f49040ad930 0x7f49040afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:10.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.152+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 <== osd.4 v2:172.21.15.92:6808/477256695 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4908018ec0 con 0x7f49040ad930 2024-04-25T20:58:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.170+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 --> [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4918063550 con 0x7f49040ad930 2024-04-25T20:58:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f490ffff700 1 -- 172.21.15.47:0/1675102035 <== osd.4 v2:172.21.15.92:6808/477256695 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4918063550 con 0x7f49040ad930 2024-04-25T20:58:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 >> [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] conn(0x7f49040ad930 msgr2=0x7f49040afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f491d429700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] conn(0x7f49040ad930 0x7f49040afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f49040a3d50 msgr2=0x7f49040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f491d429700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f49040a3d50 0x7f49040a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f490000b5c0 tx=0x7f4900005960 comp rx=0 tx=0).stop 2024-04-25T20:58:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 msgr2=0x7f4918101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f491d429700 1 --2- 172.21.15.47:0/1675102035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491806a750 0x7f4918101a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f490800b530 tx=0x7f490800b840 comp rx=0 tx=0).stop 2024-04-25T20:58:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f49177fe700 1 -- 172.21.15.47:0/1675102035 reap_dead start 2024-04-25T20:58:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 shutdown_connections 2024-04-25T20:58:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.171+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 >> 172.21.15.47:0/1675102035 conn(0x7f49180788f0 msgr2=0x7f49180ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.172+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 shutdown_connections 2024-04-25T20:58:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:10.172+0000 7f491d429700 1 -- 172.21.15.47:0/1675102035 wait complete. 2024-04-25T20:58:10.647 INFO:teuthology.orchestra.run.smithi047.stdout:223338299443 2024-04-25T20:58:10.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T20:58:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 -- 172.21.15.47:0/849572220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff904075590 msgr2=0x7ff90410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 --2- 172.21.15.47:0/849572220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff904075590 0x7ff90410ef00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009990 tx=0x7ff8f4009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 -- 172.21.15.47:0/849572220 shutdown_connections 2024-04-25T20:58:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 --2- 172.21.15.47:0/849572220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90410f440 0x7ff904111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 --2- 172.21.15.47:0/849572220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff904074c70 0x7ff904075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 --2- 172.21.15.47:0/849572220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff904075590 0x7ff90410ef00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 -- 172.21.15.47:0/849572220 >> 172.21.15.47:0/849572220 conn(0x7ff904079ba0 msgr2=0x7ff904079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 -- 172.21.15.47:0/849572220 shutdown_connections 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.690+0000 7ff90bc54700 1 -- 172.21.15.47:0/849572220 wait complete. 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90bc54700 1 Processor -- start 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90bc54700 1 -- start start 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90bc54700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff904074c70 0x7ff90419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90bc54700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff904075590 0x7ff90419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90bc54700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff90410f440 0x7ff90419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90bc54700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff904114e70 con 0x7ff90410f440 2024-04-25T20:58:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90bc54700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff904114cf0 con 0x7ff904074c70 2024-04-25T20:58:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90bc54700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff904114ff0 con 0x7ff904075590 2024-04-25T20:58:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90a1f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff90410f440 0x7ff90419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90a1f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff90410f440 0x7ff90419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53788/0 (socket says 172.21.15.47:53788) 2024-04-25T20:58:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90a1f1700 1 -- 172.21.15.47:0/2359561464 learned_addr learned my addr 172.21.15.47:0/2359561464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff9091ef700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff904075590 0x7ff90419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff9099f0700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff904074c70 0x7ff90419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90a1f1700 1 -- 172.21.15.47:0/2359561464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff904075590 msgr2=0x7ff90419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90a1f1700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff904075590 0x7ff90419b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90a1f1700 1 -- 172.21.15.47:0/2359561464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff904074c70 msgr2=0x7ff90419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90a1f1700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff904074c70 0x7ff90419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff90a1f1700 1 -- 172.21.15.47:0/2359561464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9041a03a0 con 0x7ff90410f440 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.691+0000 7ff9099f0700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff904074c70 0x7ff90419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.692+0000 7ff90a1f1700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff90410f440 0x7ff90419fc60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00ada0 tx=0x7ff8fc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.692+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc017020 con 0x7ff90410f440 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.692+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9041a0690 con 0x7ff90410f440 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.692+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8fc007e30 con 0x7ff90410f440 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.692+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc0053e0 con 0x7ff90410f440 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.692+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041a7f10 con 0x7ff90410f440 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.693+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff904063190 con 0x7ff90410f440 2024-04-25T20:58:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.694+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff8fc00c980 con 0x7ff90410f440 2024-04-25T20:58:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.695+0000 7ff8faffd700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8f00a3cd0 0x7ff8f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.695+0000 7ff9099f0700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8f00a3cd0 0x7ff8f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.695+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7ff8fc0d0810 con 0x7ff90410f440 2024-04-25T20:58:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.695+0000 7ff8faffd700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] conn(0x7ff8f00ad8b0 0x7ff8f00afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.695+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 --> [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8fc00fe50 con 0x7ff8f00ad8b0 2024-04-25T20:58:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.695+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff8fc0d0bc0 con 0x7ff90410f440 2024-04-25T20:58:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.695+0000 7ff9099f0700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8f00a3cd0 0x7ff8f00a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff900007900 tx=0x7ff900008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.696+0000 7ff9091ef700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] conn(0x7ff8f00ad8b0 0x7ff8f00afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.696+0000 7ff9091ef700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] conn(0x7ff8f00ad8b0 0x7ff8f00afd30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.697+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 <== osd.5 v2:172.21.15.186:6808/2592795959 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8fc00fe50 con 0x7ff8f00ad8b0 2024-04-25T20:58:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.714+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 --> [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff904063610 con 0x7ff8f00ad8b0 2024-04-25T20:58:11.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.714+0000 7ff8faffd700 1 -- 172.21.15.47:0/2359561464 <== osd.5 v2:172.21.15.186:6808/2592795959 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff904063610 con 0x7ff8f00ad8b0 2024-04-25T20:58:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 >> [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] conn(0x7ff8f00ad8b0 msgr2=0x7ff8f00afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] conn(0x7ff8f00ad8b0 0x7ff8f00afd30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8f00a3cd0 msgr2=0x7ff8f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8f00a3cd0 0x7ff8f00a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff900007900 tx=0x7ff900008040 comp rx=0 tx=0).stop 2024-04-25T20:58:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff90410f440 msgr2=0x7ff90419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 --2- 172.21.15.47:0/2359561464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff90410f440 0x7ff90419fc60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00ada0 tx=0x7ff8fc00c330 comp rx=0 tx=0).stop 2024-04-25T20:58:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90a1f1700 1 -- 172.21.15.47:0/2359561464 reap_dead start 2024-04-25T20:58:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 shutdown_connections 2024-04-25T20:58:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 >> 172.21.15.47:0/2359561464 conn(0x7ff904079ba0 msgr2=0x7ff9040ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 shutdown_connections 2024-04-25T20:58:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:11.715+0000 7ff90bc54700 1 -- 172.21.15.47:0/2359561464 wait complete. 2024-04-25T20:58:11.846 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:11 smithi047 ceph-mon[29195]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:11.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:11 smithi186 ceph-mon[32529]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:12.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:11 smithi092 ceph-mon[32524]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:12.478 INFO:teuthology.orchestra.run.smithi047.stdout:167503724606 2024-04-25T20:58:12.478 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T20:58:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 -- 172.21.15.47:0/455118918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 msgr2=0x7f3f84101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 --2- 172.21.15.47:0/455118918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 0x7f3f84101490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009990 tx=0x7f3f78009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 -- 172.21.15.47:0/455118918 shutdown_connections 2024-04-25T20:58:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 --2- 172.21.15.47:0/455118918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f841019d0 0x7f3f8410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 --2- 172.21.15.47:0/455118918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f84108ad0 0x7f3f84108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 --2- 172.21.15.47:0/455118918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 0x7f3f84101490 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 -- 172.21.15.47:0/455118918 >> 172.21.15.47:0/455118918 conn(0x7f3f84077fb0 msgr2=0x7f3f840ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 -- 172.21.15.47:0/455118918 shutdown_connections 2024-04-25T20:58:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.900+0000 7f3f88a5e700 1 -- 172.21.15.47:0/455118918 wait complete. 2024-04-25T20:58:12.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f88a5e700 1 Processor -- start 2024-04-25T20:58:12.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f88a5e700 1 -- start start 2024-04-25T20:58:12.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f88a5e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 0x7f3f84102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f88a5e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f841019d0 0x7f3f84102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f88a5e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f84108ad0 0x7f3f84102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f88a5e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84112be0 con 0x7f3f84101030 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f88a5e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84112a60 con 0x7f3f841019d0 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f88a5e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84112d60 con 0x7f3f84108ad0 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.901+0000 7f3f8259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 0x7f3f84102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f8259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 0x7f3f84102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53804/0 (socket says 172.21.15.47:53804) 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f8259c700 1 -- 172.21.15.47:0/3955601408 learned_addr learned my addr 172.21.15.47:0/3955601408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f841019d0 0x7f3f84102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f82d9d700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f84108ad0 0x7f3f84102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f8259c700 1 -- 172.21.15.47:0/3955601408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f84108ad0 msgr2=0x7f3f84102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f8259c700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f84108ad0 0x7f3f84102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f8259c700 1 -- 172.21.15.47:0/3955601408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f841019d0 msgr2=0x7f3f84102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f8259c700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f841019d0 0x7f3f84102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f8259c700 1 -- 172.21.15.47:0/3955601408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f841a7620 con 0x7f3f84101030 2024-04-25T20:58:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f8259c700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 0x7f3f84102210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400c920 tx=0x7f3f7400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74007b00 con 0x7f3f84101030 2024-04-25T20:58:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f841a7910 con 0x7f3f84101030 2024-04-25T20:58:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f74004ce0 con 0x7f3f84101030 2024-04-25T20:58:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74005500 con 0x7f3f84101030 2024-04-25T20:58:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.902+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f841a7e70 con 0x7f3f84101030 2024-04-25T20:58:12.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.903+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f840630d0 con 0x7f3f84101030 2024-04-25T20:58:12.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.905+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3f74007ca0 con 0x7f3f84101030 2024-04-25T20:58:12.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.905+0000 7f3f737fe700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f6c0a3cd0 0x7f3f6c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.905+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f3f740d06a0 con 0x7f3f84101030 2024-04-25T20:58:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.905+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f6c0a3cd0 0x7f3f6c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.906+0000 7f3f737fe700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] conn(0x7f3f6c0ad8b0 0x7f3f6c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.906+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 --> [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f6c0b0400 con 0x7f3f6c0ad8b0 2024-04-25T20:58:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.906+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3f740d0af0 con 0x7f3f84101030 2024-04-25T20:58:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.906+0000 7f3f82d9d700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] conn(0x7f3f6c0ad8b0 0x7f3f6c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.906+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f6c0a3cd0 0x7f3f6c0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b5c0 tx=0x7f3f78005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.907+0000 7f3f82d9d700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] conn(0x7f3f6c0ad8b0 0x7f3f6c0afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.907+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 <== osd.10 v2:172.21.15.92:6824/3185760943 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f6c0b0400 con 0x7f3f6c0ad8b0 2024-04-25T20:58:12.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.925+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 --> [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f84069730 con 0x7f3f6c0ad8b0 2024-04-25T20:58:12.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.925+0000 7f3f737fe700 1 -- 172.21.15.47:0/3955601408 <== osd.10 v2:172.21.15.92:6824/3185760943 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3f84069730 con 0x7f3f6c0ad8b0 2024-04-25T20:58:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 >> [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] conn(0x7f3f6c0ad8b0 msgr2=0x7f3f6c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] conn(0x7f3f6c0ad8b0 0x7f3f6c0afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f6c0a3cd0 msgr2=0x7f3f6c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f3f6c0a3cd0 0x7f3f6c0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b5c0 tx=0x7f3f78005960 comp rx=0 tx=0).stop 2024-04-25T20:58:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 msgr2=0x7f3f84102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 --2- 172.21.15.47:0/3955601408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84101030 0x7f3f84102210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400c920 tx=0x7f3f7400cc30 comp rx=0 tx=0).stop 2024-04-25T20:58:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f82d9d700 1 -- 172.21.15.47:0/3955601408 reap_dead start 2024-04-25T20:58:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 shutdown_connections 2024-04-25T20:58:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 >> 172.21.15.47:0/3955601408 conn(0x7f3f84077fb0 msgr2=0x7f3f840ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 shutdown_connections 2024-04-25T20:58:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:12.926+0000 7f3f88a5e700 1 -- 172.21.15.47:0/3955601408 wait complete. 2024-04-25T20:58:13.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 -- 172.21.15.47:0/2058110913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 msgr2=0x7f59b810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 --2- 172.21.15.47:0/2058110913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 0x7f59b810e7f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f59b000b0a0 tx=0x7f59b000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 -- 172.21.15.47:0/2058110913 shutdown_connections 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 --2- 172.21.15.47:0/2058110913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 0x7f59b810e7f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 --2- 172.21.15.47:0/2058110913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b8107990 0x7f59b8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 --2- 172.21.15.47:0/2058110913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59b8101940 0x7f59b8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 -- 172.21.15.47:0/2058110913 >> 172.21.15.47:0/2058110913 conn(0x7f59b80788f0 msgr2=0x7f59b8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 -- 172.21.15.47:0/2058110913 shutdown_connections 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.101+0000 7f59bfbb9700 1 -- 172.21.15.47:0/2058110913 wait complete. 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bfbb9700 1 Processor -- start 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bfbb9700 1 -- start start 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bfbb9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59b8101940 0x7f59b8199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bfbb9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 0x7f59b81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bfbb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b8107990 0x7f59b819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bfbb9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8111af0 con 0x7f59b8102300 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bfbb9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8111970 con 0x7f59b8107990 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bfbb9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8111c70 con 0x7f59b8101940 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd154700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 0x7f59b81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd154700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 0x7f59b81995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53832/0 (socket says 172.21.15.47:53832) 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd154700 1 -- 172.21.15.47:0/469913362 learned_addr learned my addr 172.21.15.47:0/469913362 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd955700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59b8101940 0x7f59b8199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59be156700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b8107990 0x7f59b819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd154700 1 -- 172.21.15.47:0/469913362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59b8101940 msgr2=0x7f59b8199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd154700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59b8101940 0x7f59b8199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd154700 1 -- 172.21.15.47:0/469913362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b8107990 msgr2=0x7f59b819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd154700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b8107990 0x7f59b819d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59bd154700 1 -- 172.21.15.47:0/469913362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b819e080 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.102+0000 7f59be156700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b8107990 0x7f59b819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.103+0000 7f59bd154700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 0x7f59b81995b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c970 tx=0x7f59a800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.103+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8007aa0 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.103+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b819e370 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.103+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59a8004ce0 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.103+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8005510 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.103+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b81a5bf0 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.104+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59b80631d0 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.105+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f59a8007c40 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.105+0000 7f59aeffd700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a40a3cd0 0x7f59a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.106+0000 7f59bd955700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a40a3cd0 0x7f59a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.106+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f59a80d0790 con 0x7f59b8102300 2024-04-25T20:58:13.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.106+0000 7f59aeffd700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] conn(0x7f59a40ad8b0 0x7f59a40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.106+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 --> [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59a800eec0 con 0x7f59a40ad8b0 2024-04-25T20:58:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.106+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f59a80d0be0 con 0x7f59b8102300 2024-04-25T20:58:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.106+0000 7f59be156700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] conn(0x7f59a40ad8b0 0x7f59a40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.106+0000 7f59bd955700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a40a3cd0 0x7f59a40a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005950 tx=0x7f59b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.106+0000 7f59be156700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] conn(0x7f59a40ad8b0 0x7f59a40afd30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.107+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 <== osd.9 v2:172.21.15.47:6826/3118226797 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59a800eec0 con 0x7f59a40ad8b0 2024-04-25T20:58:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.124+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 --> [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59b8063650 con 0x7f59a40ad8b0 2024-04-25T20:58:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.124+0000 7f59aeffd700 1 -- 172.21.15.47:0/469913362 <== osd.9 v2:172.21.15.47:6826/3118226797 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f59b8063650 con 0x7f59a40ad8b0 2024-04-25T20:58:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] conn(0x7f59a40ad8b0 msgr2=0x7f59a40afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] conn(0x7f59a40ad8b0 0x7f59a40afd30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a40a3cd0 msgr2=0x7f59a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a40a3cd0 0x7f59a40a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005950 tx=0x7f59b400b410 comp rx=0 tx=0).stop 2024-04-25T20:58:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 msgr2=0x7f59b81995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 --2- 172.21.15.47:0/469913362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b8102300 0x7f59b81995b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c970 tx=0x7f59a800cc80 comp rx=0 tx=0).stop 2024-04-25T20:58:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59be156700 1 -- 172.21.15.47:0/469913362 reap_dead start 2024-04-25T20:58:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 shutdown_connections 2024-04-25T20:58:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 >> 172.21.15.47:0/469913362 conn(0x7f59b80788f0 msgr2=0x7f59b806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 shutdown_connections 2024-04-25T20:58:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:13.125+0000 7f59bfbb9700 1 -- 172.21.15.47:0/469913362 wait complete. 2024-04-25T20:58:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:13 smithi047 ceph-mon[29195]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T20:58:13.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:13 smithi186 ceph-mon[32529]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T20:58:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:13 smithi092 ceph-mon[32524]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T20:58:14.656 INFO:teuthology.orchestra.run.smithi047.stdout:55834574926 2024-04-25T20:58:14.657 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T20:58:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 -- 172.21.15.47:0/4054573972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668102ac0 msgr2=0x7f5668102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 --2- 172.21.15.47:0/4054573972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668102ac0 0x7f5668102f40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f565c009990 tx=0x7f565c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 -- 172.21.15.47:0/4054573972 shutdown_connections 2024-04-25T20:58:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 --2- 172.21.15.47:0/4054573972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5668103480 0x7f566810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 --2- 172.21.15.47:0/4054573972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668108ab0 0x7f5668108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 --2- 172.21.15.47:0/4054573972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668102ac0 0x7f5668102f40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 -- 172.21.15.47:0/4054573972 >> 172.21.15.47:0/4054573972 conn(0x7f5668079bf0 msgr2=0x7f566807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 -- 172.21.15.47:0/4054573972 shutdown_connections 2024-04-25T20:58:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.090+0000 7f566f741700 1 -- 172.21.15.47:0/4054573972 wait complete. 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566f741700 1 Processor -- start 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566f741700 1 -- start start 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566f741700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668102ac0 0x7f56681991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566f741700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668103480 0x7f56681996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566f741700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5668108ab0 0x7f566819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566f741700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5668112cf0 con 0x7f5668103480 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566f741700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5668112b70 con 0x7f5668102ac0 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566f741700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5668112e70 con 0x7f5668108ab0 2024-04-25T20:58:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566ccdc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668103480 0x7f56681996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566ccdc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668103480 0x7f56681996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33406/0 (socket says 172.21.15.47:33406) 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566ccdc700 1 -- 172.21.15.47:0/746087889 learned_addr learned my addr 172.21.15.47:0/746087889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566dcde700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5668108ab0 0x7f566819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.091+0000 7f566d4dd700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668102ac0 0x7f56681991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566ccdc700 1 -- 172.21.15.47:0/746087889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5668108ab0 msgr2=0x7f566819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566ccdc700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5668108ab0 0x7f566819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566ccdc700 1 -- 172.21.15.47:0/746087889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668102ac0 msgr2=0x7f56681991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566ccdc700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668102ac0 0x7f56681991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566ccdc700 1 -- 172.21.15.47:0/746087889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566819e1b0 con 0x7f5668103480 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566d4dd700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668102ac0 0x7f56681991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566ccdc700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668103480 0x7f56681996e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f565c000c00 tx=0x7f565c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c027070 con 0x7f5668103480 2024-04-25T20:58:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566819e440 con 0x7f5668103480 2024-04-25T20:58:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c003f20 con 0x7f5668103480 2024-04-25T20:58:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c022cf0 con 0x7f5668103480 2024-04-25T20:58:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.092+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681a5d20 con 0x7f5668103480 2024-04-25T20:58:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.093+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5668063150 con 0x7f5668103480 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.094+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f565c02a070 con 0x7f5668103480 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.095+0000 7f56667fc700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56500a3cd0 0x7f56500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.095+0000 7f566d4dd700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56500a3cd0 0x7f56500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.095+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f565c0d1910 con 0x7f5668103480 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.095+0000 7f56667fc700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] conn(0x7f56500ad8b0 0x7f56500afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.095+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 --> [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f565c0095d0 con 0x7f56500ad8b0 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.095+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f565c0d1d60 con 0x7f5668103480 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.095+0000 7f566d4dd700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56500a3cd0 0x7f56500a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5660009fa0 tx=0x7f5660009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:15.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.095+0000 7f566dcde700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] conn(0x7f56500ad8b0 0x7f56500afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.096+0000 7f566dcde700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] conn(0x7f56500ad8b0 0x7f56500afd30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:15.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.097+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 <== osd.2 v2:172.21.15.186:6800/651738121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f565c0095d0 con 0x7f56500ad8b0 2024-04-25T20:58:15.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.113+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 --> [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f56680635d0 con 0x7f56500ad8b0 2024-04-25T20:58:15.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f56667fc700 1 -- 172.21.15.47:0/746087889 <== osd.2 v2:172.21.15.186:6800/651738121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f56680635d0 con 0x7f56500ad8b0 2024-04-25T20:58:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 >> [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] conn(0x7f56500ad8b0 msgr2=0x7f56500afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] conn(0x7f56500ad8b0 0x7f56500afd30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56500a3cd0 msgr2=0x7f56500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56500a3cd0 0x7f56500a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5660009fa0 tx=0x7f5660009570 comp rx=0 tx=0).stop 2024-04-25T20:58:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668103480 msgr2=0x7f56681996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 --2- 172.21.15.47:0/746087889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5668103480 0x7f56681996e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f565c000c00 tx=0x7f565c0047b0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566dcde700 1 -- 172.21.15.47:0/746087889 reap_dead start 2024-04-25T20:58:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 shutdown_connections 2024-04-25T20:58:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 >> 172.21.15.47:0/746087889 conn(0x7f5668079bf0 msgr2=0x7f5668100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 shutdown_connections 2024-04-25T20:58:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.114+0000 7f566f741700 1 -- 172.21.15.47:0/746087889 wait complete. 2024-04-25T20:58:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 -- 172.21.15.47:0/4196229013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc78101930 msgr2=0x7fbc78101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 --2- 172.21.15.47:0/4196229013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc78101930 0x7fbc78101db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009990 tx=0x7fbc6c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 -- 172.21.15.47:0/4196229013 shutdown_connections 2024-04-25T20:58:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 --2- 172.21.15.47:0/4196229013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc781022f0 0x7fbc7810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 --2- 172.21.15.47:0/4196229013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc78107980 0x7fbc78107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 --2- 172.21.15.47:0/4196229013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc78101930 0x7fbc78101db0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 -- 172.21.15.47:0/4196229013 >> 172.21.15.47:0/4196229013 conn(0x7fbc780788b0 msgr2=0x7fbc78078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 -- 172.21.15.47:0/4196229013 shutdown_connections 2024-04-25T20:58:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.227+0000 7fbc7dc39700 1 -- 172.21.15.47:0/4196229013 wait complete. 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.228+0000 7fbc7dc39700 1 Processor -- start 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.228+0000 7fbc7dc39700 1 -- start start 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc7dc39700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc78101930 0x7fbc78199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc7dc39700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781022f0 0x7fbc7819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc7dc39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc78107980 0x7fbc781a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc7dc39700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111ad0 con 0x7fbc781022f0 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc7dc39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111950 con 0x7fbc78107980 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc7dc39700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111c50 con 0x7fbc78101930 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc76ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781022f0 0x7fbc7819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc76ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781022f0 0x7fbc7819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33432/0 (socket says 172.21.15.47:33432) 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc76ffd700 1 -- 172.21.15.47:0/2373388861 learned_addr learned my addr 172.21.15.47:0/2373388861 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc77fff700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc78107980 0x7fbc781a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc76ffd700 1 -- 172.21.15.47:0/2373388861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc78101930 msgr2=0x7fbc78199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc76ffd700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc78101930 0x7fbc78199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc76ffd700 1 -- 172.21.15.47:0/2373388861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc78107980 msgr2=0x7fbc781a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc76ffd700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc78107980 0x7fbc781a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.229+0000 7fbc76ffd700 1 -- 172.21.15.47:0/2373388861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc781a5410 con 0x7fbc781022f0 2024-04-25T20:58:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.230+0000 7fbc76ffd700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781022f0 0x7fbc7819a390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c000c00 tx=0x7fbc6c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.230+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c027070 con 0x7fbc781022f0 2024-04-25T20:58:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.230+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc6c0041d0 con 0x7fbc781022f0 2024-04-25T20:58:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.230+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc781a55e0 con 0x7fbc781022f0 2024-04-25T20:58:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.231+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c00f460 con 0x7fbc781022f0 2024-04-25T20:58:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.231+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc781a5a90 con 0x7fbc781022f0 2024-04-25T20:58:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.232+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc78063150 con 0x7fbc781022f0 2024-04-25T20:58:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.233+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbc6c02a070 con 0x7fbc781022f0 2024-04-25T20:58:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.233+0000 7fbc74ff9700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbc600a3d50 0x7fbc600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.234+0000 7fbc777fe700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbc600a3d50 0x7fbc600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.234+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fbc6c0d1500 con 0x7fbc781022f0 2024-04-25T20:58:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.234+0000 7fbc74ff9700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] conn(0x7fbc600ad930 0x7fbc600afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.234+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 --> [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc6c0043d0 con 0x7fbc600ad930 2024-04-25T20:58:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.234+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c0d8050 con 0x7fbc781022f0 2024-04-25T20:58:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.234+0000 7fbc777fe700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbc600a3d50 0x7fbc600a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800a9b0 tx=0x7fbc68005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.234+0000 7fbc77fff700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] conn(0x7fbc600ad930 0x7fbc600afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.235+0000 7fbc77fff700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] conn(0x7fbc600ad930 0x7fbc600afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.235+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 <== osd.8 v2:172.21.15.186:6816/773299526 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc6c0043d0 con 0x7fbc600ad930 2024-04-25T20:58:15.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.253+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 --> [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbc78063600 con 0x7fbc600ad930 2024-04-25T20:58:15.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.253+0000 7fbc74ff9700 1 -- 172.21.15.47:0/2373388861 <== osd.8 v2:172.21.15.186:6816/773299526 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbc78063600 con 0x7fbc600ad930 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 >> [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] conn(0x7fbc600ad930 msgr2=0x7fbc600afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] conn(0x7fbc600ad930 0x7fbc600afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbc600a3d50 msgr2=0x7fbc600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbc600a3d50 0x7fbc600a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800a9b0 tx=0x7fbc68005c90 comp rx=0 tx=0).stop 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781022f0 msgr2=0x7fbc7819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 --2- 172.21.15.47:0/2373388861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781022f0 0x7fbc7819a390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c000c00 tx=0x7fbc6c0047b0 comp rx=0 tx=0).stop 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc77fff700 1 -- 172.21.15.47:0/2373388861 reap_dead start 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 shutdown_connections 2024-04-25T20:58:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 >> 172.21.15.47:0/2373388861 conn(0x7fbc780788b0 msgr2=0x7fbc780ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:15.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 shutdown_connections 2024-04-25T20:58:15.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:15.254+0000 7fbc7dc39700 1 -- 172.21.15.47:0/2373388861 wait complete. 2024-04-25T20:58:15.870 INFO:teuthology.orchestra.run.smithi047.stdout:77309411404 2024-04-25T20:58:15.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T20:58:15.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:15 smithi047 ceph-mon[29195]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T20:58:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:15 smithi186 ceph-mon[32529]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T20:58:16.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:15 smithi092 ceph-mon[32524]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T20:58:16.779 INFO:teuthology.orchestra.run.smithi047.stdout:141733920834 2024-04-25T20:58:16.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T20:58:17.219 INFO:teuthology.orchestra.run.smithi047.stdout:257698037808 2024-04-25T20:58:17.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T20:58:17.599 INFO:teuthology.orchestra.run.smithi047.stdout:292057776170 2024-04-25T20:58:17.600 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T20:58:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:17 smithi047 ceph-mon[29195]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:17.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:17 smithi186 ceph-mon[32529]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:17 smithi092 ceph-mon[32524]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:18.090 INFO:teuthology.orchestra.run.smithi047.stdout:317827579942 2024-04-25T20:58:18.090 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T20:58:18.169 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.533 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:19.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:19 smithi047 ceph-mon[29195]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:19.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:19 smithi186 ceph-mon[32529]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:20.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:19 smithi092 ceph-mon[32524]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:21.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:21 smithi047 ceph-mon[29195]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:21.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:21 smithi186 ceph-mon[32529]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:22.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:21 smithi092 ceph-mon[32524]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:22.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 -- 172.21.15.47:0/1812959248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe18103950 msgr2=0x7fbe1806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/1812959248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe18103950 0x7fbe1806b110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009990 tx=0x7fbe08009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 -- 172.21.15.47:0/1812959248 shutdown_connections 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/1812959248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe18103950 0x7fbe1806b110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/1812959248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe1810f390 0x7fbe18111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/1812959248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe1806b650 0x7fbe1806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 -- 172.21.15.47:0/1812959248 >> 172.21.15.47:0/1812959248 conn(0x7fbe180788f0 msgr2=0x7fbe18078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 -- 172.21.15.47:0/1812959248 shutdown_connections 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.834+0000 7fbe1cc63700 1 -- 172.21.15.47:0/1812959248 wait complete. 2024-04-25T20:58:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1cc63700 1 Processor -- start 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1cc63700 1 -- start start 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1cc63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe1806b650 0x7fbe1819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1cc63700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe18103950 0x7fbe1819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1cc63700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe1810f390 0x7fbe1819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1cc63700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18114f10 con 0x7fbe1806b650 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1cc63700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18114d90 con 0x7fbe18103950 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1cc63700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18115090 con 0x7fbe1810f390 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe1806b650 0x7fbe1819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe1806b650 0x7fbe1819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33442/0 (socket says 172.21.15.47:33442) 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 -- 172.21.15.47:0/4167351250 learned_addr learned my addr 172.21.15.47:0/4167351250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 -- 172.21.15.47:0/4167351250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe1810f390 msgr2=0x7fbe1819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe1810f390 0x7fbe1819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 -- 172.21.15.47:0/4167351250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe18103950 msgr2=0x7fbe1819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe15d9b700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe18103950 0x7fbe1819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe18103950 0x7fbe1819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 -- 172.21.15.47:0/4167351250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe181a04b0 con 0x7fbe1806b650 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe15d9b700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe18103950 0x7fbe1819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.835+0000 7fbe1659c700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe1806b650 0x7fbe1819b4a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe080057f0 tx=0x7fbe08003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.836+0000 7fbe077fe700 1 -- 172.21.15.47:0/4167351250 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08027070 con 0x7fbe1806b650 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.836+0000 7fbe077fe700 1 -- 172.21.15.47:0/4167351250 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe080041c0 con 0x7fbe1806b650 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.836+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe181a0740 con 0x7fbe1806b650 2024-04-25T20:58:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.836+0000 7fbe077fe700 1 -- 172.21.15.47:0/4167351250 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08022e60 con 0x7fbe1806b650 2024-04-25T20:58:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.836+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe181a8020 con 0x7fbe1806b650 2024-04-25T20:58:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.837+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe18101f80 con 0x7fbe1806b650 2024-04-25T20:58:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.838+0000 7fbe077fe700 1 -- 172.21.15.47:0/4167351250 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe0802a070 con 0x7fbe1806b650 2024-04-25T20:58:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.839+0000 7fbe077fe700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbe000a3cd0 0x7fbe000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.839+0000 7fbe15d9b700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbe000a3cd0 0x7fbe000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.839+0000 7fbe077fe700 1 -- 172.21.15.47:0/4167351250 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fbe080d12b0 con 0x7fbe1806b650 2024-04-25T20:58:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.839+0000 7fbe15d9b700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbe000a3cd0 0x7fbe000a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe180791f0 tx=0x7fbe0c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:22.844+0000 7fbe077fe700 1 -- 172.21.15.47:0/4167351250 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe08013070 con 0x7fbe1806b650 2024-04-25T20:58:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.056+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbe181a82b0 con 0x7fbe1806b650 2024-04-25T20:58:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.056+0000 7fbe077fe700 1 -- 172.21.15.47:0/4167351250 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbe0809dbb0 con 0x7fbe1806b650 2024-04-25T20:58:23.058 INFO:teuthology.orchestra.run.smithi047.stdout:201863462971 2024-04-25T20:58:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.058+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbe000a3cd0 msgr2=0x7fbe000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.058+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbe000a3cd0 0x7fbe000a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe180791f0 tx=0x7fbe0c008040 comp rx=0 tx=0).stop 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.058+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe1806b650 msgr2=0x7fbe1819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe1806b650 0x7fbe1819b4a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe080057f0 tx=0x7fbe08003730 comp rx=0 tx=0).stop 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 shutdown_connections 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fbe000a3cd0 0x7fbe000a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe1806b650 0x7fbe1819b4a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe1810f390 0x7fbe1819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 --2- 172.21.15.47:0/4167351250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe18103950 0x7fbe1819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 >> 172.21.15.47:0/4167351250 conn(0x7fbe180788f0 msgr2=0x7fbe18100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 shutdown_connections 2024-04-25T20:58:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:23.059+0000 7fbe1cc63700 1 -- 172.21.15.47:0/4167351250 wait complete. 2024-04-25T20:58:23.936 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:23 smithi047 ceph-mon[29195]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:23.936 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:23 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/4167351250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T20:58:23.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:23 smithi186 ceph-mon[32529]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:23.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:23 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/4167351250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T20:58:24.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:23 smithi092 ceph-mon[32524]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:24.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:23 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/4167351250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T20:58:25.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:25 smithi047 ceph-mon[29195]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:25.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:25 smithi186 ceph-mon[32529]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:25 smithi092 ceph-mon[32524]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:27.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/3931527771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58108a70 msgr2=0x7f1f58108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:27.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/3931527771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58108a70 0x7f1f58108e50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800b0a0 tx=0x7f1f4800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/3931527771 shutdown_connections 2024-04-25T20:58:27.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/3931527771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58108a70 0x7f1f58108e50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/3931527771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f58103480 0x7f1f5810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/3931527771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f58102ac0 0x7f1f58102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/3931527771 >> 172.21.15.47:0/3931527771 conn(0x7f1f58079bb0 msgr2=0x7f1f58079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/3931527771 shutdown_connections 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.075+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/3931527771 wait complete. 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5cc5f700 1 Processor -- start 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5cc5f700 1 -- start start 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5cc5f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58102ac0 0x7f1f58199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5cc5f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f58103480 0x7f1f58199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5cc5f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f58108a70 0x7f1f5819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5cc5f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58112c50 con 0x7f1f58102ac0 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5cc5f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58112ad0 con 0x7f1f58103480 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5cc5f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58112dd0 con 0x7f1f58108a70 2024-04-25T20:58:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58102ac0 0x7f1f58199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58102ac0 0x7f1f58199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58748/0 (socket says 172.21.15.47:58748) 2024-04-25T20:58:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5659c700 1 -- 172.21.15.47:0/76215616 learned_addr learned my addr 172.21.15.47:0/76215616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5659c700 1 -- 172.21.15.47:0/76215616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f58108a70 msgr2=0x7f1f5819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5659c700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f58108a70 0x7f1f5819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5659c700 1 -- 172.21.15.47:0/76215616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f58103480 msgr2=0x7f1f58199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5659c700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f58103480 0x7f1f58199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.076+0000 7f1f5659c700 1 -- 172.21.15.47:0/76215616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5819e030 con 0x7f1f58102ac0 2024-04-25T20:58:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.077+0000 7f1f5659c700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58102ac0 0x7f1f58199020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800b070 tx=0x7f1f48004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.077+0000 7f1f477fe700 1 -- 172.21.15.47:0/76215616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48021070 con 0x7f1f58102ac0 2024-04-25T20:58:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.077+0000 7f1f477fe700 1 -- 172.21.15.47:0/76215616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f48007db0 con 0x7f1f58102ac0 2024-04-25T20:58:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.077+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5819e2c0 con 0x7f1f58102ac0 2024-04-25T20:58:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.077+0000 7f1f477fe700 1 -- 172.21.15.47:0/76215616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4801d450 con 0x7f1f58102ac0 2024-04-25T20:58:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.077+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581a5ba0 con 0x7f1f58102ac0 2024-04-25T20:58:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.078+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f58069730 con 0x7f1f58102ac0 2024-04-25T20:58:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.079+0000 7f1f477fe700 1 -- 172.21.15.47:0/76215616 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1f480040b0 con 0x7f1f58102ac0 2024-04-25T20:58:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.080+0000 7f1f477fe700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f1f400a3cd0 0x7f1f400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.080+0000 7f1f55d9b700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f1f400a3cd0 0x7f1f400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.080+0000 7f1f477fe700 1 -- 172.21.15.47:0/76215616 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f1f4800f020 con 0x7f1f58102ac0 2024-04-25T20:58:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.080+0000 7f1f55d9b700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f1f400a3cd0 0x7f1f400a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005fd0 tx=0x7f1f4c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:27.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.084+0000 7f1f477fe700 1 -- 172.21.15.47:0/76215616 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f4802a020 con 0x7f1f58102ac0 2024-04-25T20:58:27.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.300+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1f5802df80 con 0x7f1f58102ac0 2024-04-25T20:58:27.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.300+0000 7f1f477fe700 1 -- 172.21.15.47:0/76215616 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1f4809e5e0 con 0x7f1f58102ac0 2024-04-25T20:58:27.302 INFO:teuthology.orchestra.run.smithi047.stdout:223338299446 2024-04-25T20:58:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f1f400a3cd0 msgr2=0x7f1f400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f1f400a3cd0 0x7f1f400a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005fd0 tx=0x7f1f4c005d00 comp rx=0 tx=0).stop 2024-04-25T20:58:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58102ac0 msgr2=0x7f1f58199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58102ac0 0x7f1f58199020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800b070 tx=0x7f1f48004ef0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 shutdown_connections 2024-04-25T20:58:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f1f400a3cd0 0x7f1f400a6190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f58102ac0 0x7f1f58199020 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f58108a70 0x7f1f5819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 --2- 172.21.15.47:0/76215616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f58103480 0x7f1f58199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 >> 172.21.15.47:0/76215616 conn(0x7f1f58079bb0 msgr2=0x7f1f58101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.302+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 shutdown_connections 2024-04-25T20:58:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:27.303+0000 7f1f5cc5f700 1 -- 172.21.15.47:0/76215616 wait complete. 2024-04-25T20:58:27.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:27 smithi186 ceph-mon[32529]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T20:58:27.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:27 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/76215616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T20:58:28.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:27 smithi092 ceph-mon[32524]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T20:58:28.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:27 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/76215616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T20:58:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:27 smithi047 ceph-mon[29195]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T20:58:28.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:27 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/76215616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T20:58:29.778 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462966 got 201863462971 for osd.6 2024-04-25T20:58:29.779 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:29.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:29 smithi047 ceph-mon[29195]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:29.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:29 smithi186 ceph-mon[32529]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:30.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:29 smithi092 ceph-mon[32524]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 -- 172.21.15.47:0/1986495793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 msgr2=0x7f1530108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 --2- 172.21.15.47:0/1986495793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 0x7f1530108ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1520009990 tx=0x7f1520009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 -- 172.21.15.47:0/1986495793 shutdown_connections 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 --2- 172.21.15.47:0/1986495793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 0x7f1530108ea0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 --2- 172.21.15.47:0/1986495793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f153006bad0 0x7f153010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 --2- 172.21.15.47:0/1986495793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153006b110 0x7f153006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 -- 172.21.15.47:0/1986495793 >> 172.21.15.47:0/1986495793 conn(0x7f15300788f0 msgr2=0x7f1530078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 -- 172.21.15.47:0/1986495793 shutdown_connections 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.570+0000 7f1535b9b700 1 -- 172.21.15.47:0/1986495793 wait complete. 2024-04-25T20:58:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.571+0000 7f1535b9b700 1 Processor -- start 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.571+0000 7f1535b9b700 1 -- start start 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.571+0000 7f1535b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153006b110 0x7f1530074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.571+0000 7f1535b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f153006bad0 0x7f153006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.571+0000 7f1535b9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 0x7f153006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.571+0000 7f1535b9b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1530112c10 con 0x7f1530108ac0 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.571+0000 7f1535b9b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1530112a90 con 0x7f153006b110 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.571+0000 7f1535b9b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1530112d90 con 0x7f153006bad0 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 0x7f153006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 0x7f153006d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58776/0 (socket says 172.21.15.47:58776) 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 -- 172.21.15.47:0/328113619 learned_addr learned my addr 172.21.15.47:0/328113619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152f7fe700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153006b110 0x7f1530074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152effd700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f153006bad0 0x7f153006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 -- 172.21.15.47:0/328113619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f153006bad0 msgr2=0x7f153006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f153006bad0 0x7f153006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 -- 172.21.15.47:0/328113619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153006b110 msgr2=0x7f1530074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153006b110 0x7f1530074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 -- 172.21.15.47:0/328113619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153006dfc0 con 0x7f1530108ac0 2024-04-25T20:58:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152f7fe700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153006b110 0x7f1530074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152ffff700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 0x7f153006d8e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f152800ada0 tx=0x7f152800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152cff9700 1 -- 172.21.15.47:0/328113619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528017020 con 0x7f1530108ac0 2024-04-25T20:58:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152cff9700 1 -- 172.21.15.47:0/328113619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f152800cd80 con 0x7f1530108ac0 2024-04-25T20:58:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f152cff9700 1 -- 172.21.15.47:0/328113619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528005260 con 0x7f1530108ac0 2024-04-25T20:58:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.572+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1530071e40 con 0x7f1530108ac0 2024-04-25T20:58:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.573+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1530072330 con 0x7f1530108ac0 2024-04-25T20:58:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.574+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153006a9c0 con 0x7f1530108ac0 2024-04-25T20:58:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.575+0000 7f152cff9700 1 -- 172.21.15.47:0/328113619 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f152800c8d0 con 0x7f1530108ac0 2024-04-25T20:58:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.575+0000 7f152cff9700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f151c0a3cd0 0x7f151c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:31.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.575+0000 7f152f7fe700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f151c0a3cd0 0x7f151c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:31.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.575+0000 7f152cff9700 1 -- 172.21.15.47:0/328113619 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f15280cfc00 con 0x7f1530108ac0 2024-04-25T20:58:31.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.576+0000 7f152f7fe700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f151c0a3cd0 0x7f151c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f152000b5c0 tx=0x7f1520005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:31.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.580+0000 7f152cff9700 1 -- 172.21.15.47:0/328113619 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1528021020 con 0x7f1530108ac0 2024-04-25T20:58:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.792+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f153006e8d0 con 0x7f1530108ac0 2024-04-25T20:58:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.792+0000 7f152cff9700 1 -- 172.21.15.47:0/328113619 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f152809c500 con 0x7f1530108ac0 2024-04-25T20:58:31.794 INFO:teuthology.orchestra.run.smithi047.stdout:167503724610 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.795+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f151c0a3cd0 msgr2=0x7f151c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.795+0000 7f1535b9b700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f151c0a3cd0 0x7f151c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f152000b5c0 tx=0x7f1520005960 comp rx=0 tx=0).stop 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.795+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 msgr2=0x7f153006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.795+0000 7f1535b9b700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 0x7f153006d8e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f152800ada0 tx=0x7f152800c330 comp rx=0 tx=0).stop 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.796+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 shutdown_connections 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.796+0000 7f1535b9b700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f151c0a3cd0 0x7f151c0a6190 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f152000b5c0 tx=0x7f1520005960 comp rx=0 tx=0).stop 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.796+0000 7f1535b9b700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1530108ac0 0x7f153006d8e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.796+0000 7f1535b9b700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153006b110 0x7f1530074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.796+0000 7f1535b9b700 1 --2- 172.21.15.47:0/328113619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f153006bad0 0x7f153006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.796+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 >> 172.21.15.47:0/328113619 conn(0x7f15300788f0 msgr2=0x7f1530100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:31.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.796+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 shutdown_connections 2024-04-25T20:58:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:31.796+0000 7f1535b9b700 1 -- 172.21.15.47:0/328113619 wait complete. 2024-04-25T20:58:31.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:31 smithi047 ceph-mon[29195]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:31.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:31 smithi186 ceph-mon[32529]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:32.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:31 smithi092 ceph-mon[32524]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:32.802 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:32 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/328113619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T20:58:32.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:32 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/328113619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T20:58:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:32 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/328113619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T20:58:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 -- 172.21.15.47:0/1846169929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 msgr2=0x7f52e010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 --2- 172.21.15.47:0/1846169929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 0x7f52e010e7f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b0a0 tx=0x7f52d800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 -- 172.21.15.47:0/1846169929 shutdown_connections 2024-04-25T20:58:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 --2- 172.21.15.47:0/1846169929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 0x7f52e010e7f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 --2- 172.21.15.47:0/1846169929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52e0107990 0x7f52e0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 --2- 172.21.15.47:0/1846169929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0101940 0x7f52e0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 -- 172.21.15.47:0/1846169929 >> 172.21.15.47:0/1846169929 conn(0x7f52e00788f0 msgr2=0x7f52e0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 -- 172.21.15.47:0/1846169929 shutdown_connections 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.723+0000 7f52e75e3700 1 -- 172.21.15.47:0/1846169929 wait complete. 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e75e3700 1 Processor -- start 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e75e3700 1 -- start start 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e75e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52e0101940 0x7f52e0199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e75e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 0x7f52e019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e75e3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0107990 0x7f52e01a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e75e3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0111ae0 con 0x7f52e0102300 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e75e3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0111960 con 0x7f52e0101940 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e75e3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0111c60 con 0x7f52e0107990 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.724+0000 7f52e4b7e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 0x7f52e019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e4b7e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 0x7f52e019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54982/0 (socket says 172.21.15.47:54982) 2024-04-25T20:58:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e4b7e700 1 -- 172.21.15.47:0/3162756793 learned_addr learned my addr 172.21.15.47:0/3162756793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e537f700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52e0101940 0x7f52e0199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e5b80700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0107990 0x7f52e01a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e4b7e700 1 -- 172.21.15.47:0/3162756793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0107990 msgr2=0x7f52e01a4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e4b7e700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0107990 0x7f52e01a4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e4b7e700 1 -- 172.21.15.47:0/3162756793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52e0101940 msgr2=0x7f52e0199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e4b7e700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52e0101940 0x7f52e0199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e4b7e700 1 -- 172.21.15.47:0/3162756793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e01a5460 con 0x7f52e0102300 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e5b80700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0107990 0x7f52e01a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e4b7e700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 0x7f52e019a3e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52d000c9b0 tx=0x7f52d000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52d67fc700 1 -- 172.21.15.47:0/3162756793 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0007b00 con 0x7f52e0102300 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52d67fc700 1 -- 172.21.15.47:0/3162756793 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d0007ca0 con 0x7f52e0102300 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e01a5630 con 0x7f52e0102300 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52d67fc700 1 -- 172.21.15.47:0/3162756793 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0018660 con 0x7f52e0102300 2024-04-25T20:58:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.725+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e01a5bc0 con 0x7f52e0102300 2024-04-25T20:58:33.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.726+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e0069730 con 0x7f52e0102300 2024-04-25T20:58:33.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.727+0000 7f52d67fc700 1 -- 172.21.15.47:0/3162756793 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f52d000f450 con 0x7f52e0102300 2024-04-25T20:58:33.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.728+0000 7f52d67fc700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f52cc0a3d50 0x7f52cc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.728+0000 7f52d67fc700 1 -- 172.21.15.47:0/3162756793 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f52d00d0500 con 0x7f52e0102300 2024-04-25T20:58:33.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.728+0000 7f52e537f700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f52cc0a3d50 0x7f52cc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.729+0000 7f52e537f700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f52cc0a3d50 0x7f52cc0a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f52dc007b60 tx=0x7f52dc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.733+0000 7f52d67fc700 1 -- 172.21.15.47:0/3162756793 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52d0014030 con 0x7f52e0102300 2024-04-25T20:58:33.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:33 smithi047 ceph-mon[29195]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:33.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:33 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1015686168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T20:58:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.943+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f52e006ace0 con 0x7f52e0102300 2024-04-25T20:58:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.944+0000 7f52d67fc700 1 -- 172.21.15.47:0/3162756793 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f52d009cd70 con 0x7f52e0102300 2024-04-25T20:58:33.945 INFO:teuthology.orchestra.run.smithi047.stdout:347892351012 2024-04-25T20:58:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f52cc0a3d50 msgr2=0x7f52cc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f52cc0a3d50 0x7f52cc0a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f52dc007b60 tx=0x7f52dc0073d0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 msgr2=0x7f52e019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 0x7f52e019a3e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52d000c9b0 tx=0x7f52d000ccc0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 shutdown_connections 2024-04-25T20:58:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f52cc0a3d50 0x7f52cc0a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0107990 0x7f52e01a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52e0101940 0x7f52e0199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 --2- 172.21.15.47:0/3162756793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52e0102300 0x7f52e019a3e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 >> 172.21.15.47:0/3162756793 conn(0x7f52e00788f0 msgr2=0x7f52e00ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 shutdown_connections 2024-04-25T20:58:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.946+0000 7f52e75e3700 1 -- 172.21.15.47:0/3162756793 wait complete. 2024-04-25T20:58:33.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:33 smithi186 ceph-mon[32529]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:33.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:33 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1015686168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T20:58:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:33 smithi092 ceph-mon[32524]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:33 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1015686168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.392+0000 7f4ccacda700 1 -- 172.21.15.47:0/3677088451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4102320 msgr2=0x7f4cc410e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.392+0000 7f4ccacda700 1 --2- 172.21.15.47:0/3677088451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4102320 0x7f4cc410e830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00b0a0 tx=0x7f4cbc00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 -- 172.21.15.47:0/3677088451 shutdown_connections 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 --2- 172.21.15.47:0/3677088451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4102320 0x7f4cc410e830 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 --2- 172.21.15.47:0/3677088451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc4107960 0x7f4cc4107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 --2- 172.21.15.47:0/3677088451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc4101960 0x7f4cc4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 -- 172.21.15.47:0/3677088451 >> 172.21.15.47:0/3677088451 conn(0x7f4cc40789e0 msgr2=0x7f4cc4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 -- 172.21.15.47:0/3677088451 shutdown_connections 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 -- 172.21.15.47:0/3677088451 wait complete. 2024-04-25T20:58:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 Processor -- start 2024-04-25T20:58:34.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.393+0000 7f4ccacda700 1 -- start start 2024-04-25T20:58:34.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4ccacda700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc4101960 0x7f4cc40fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4ccacda700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc4102320 0x7f4cc40feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4ccacda700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4107960 0x7f4cc40ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4ccacda700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4111ba0 con 0x7f4cc4107960 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4ccacda700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4111a20 con 0x7f4cc4102320 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4ccacda700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4111d20 con 0x7f4cc4101960 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4107960 0x7f4cc40ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4107960 0x7f4cc40ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55004/0 (socket says 172.21.15.47:55004) 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 -- 172.21.15.47:0/1005566520 learned_addr learned my addr 172.21.15.47:0/1005566520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc8a76700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc4101960 0x7f4cc40fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc3fff700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc4102320 0x7f4cc40feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 -- 172.21.15.47:0/1005566520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc4101960 msgr2=0x7f4cc40fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc4101960 0x7f4cc40fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 -- 172.21.15.47:0/1005566520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc4102320 msgr2=0x7f4cc40feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc4102320 0x7f4cc40feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 -- 172.21.15.47:0/1005566520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc41a7560 con 0x7f4cc4107960 2024-04-25T20:58:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc9277700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4107960 0x7f4cc40ff410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00b830 tx=0x7f4cbc004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc1ffb700 1 -- 172.21.15.47:0/1005566520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc021070 con 0x7f4cc4107960 2024-04-25T20:58:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc41a77f0 con 0x7f4cc4107960 2024-04-25T20:58:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc1ffb700 1 -- 172.21.15.47:0/1005566520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cbc007e30 con 0x7f4cc4107960 2024-04-25T20:58:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.394+0000 7f4cc1ffb700 1 -- 172.21.15.47:0/1005566520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc004700 con 0x7f4cc4107960 2024-04-25T20:58:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.395+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc41a7ca0 con 0x7f4cc4107960 2024-04-25T20:58:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.395+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc4069730 con 0x7f4cc4107960 2024-04-25T20:58:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.397+0000 7f4cc1ffb700 1 -- 172.21.15.47:0/1005566520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4cbc003d70 con 0x7f4cc4107960 2024-04-25T20:58:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.397+0000 7f4cc1ffb700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cb40a3cd0 0x7f4cb40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.397+0000 7f4cc8a76700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cb40a3cd0 0x7f4cb40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.397+0000 7f4cc1ffb700 1 -- 172.21.15.47:0/1005566520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f4cbc00f020 con 0x7f4cc4107960 2024-04-25T20:58:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.398+0000 7f4cc8a76700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cb40a3cd0 0x7f4cb40a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005950 tx=0x7f4cb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:34.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.401+0000 7f4cc1ffb700 1 -- 172.21.15.47:0/1005566520 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cbc02a020 con 0x7f4cc4107960 2024-04-25T20:58:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.610+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4cc402e140 con 0x7f4cc4107960 2024-04-25T20:58:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.611+0000 7f4cc1ffb700 1 -- 172.21.15.47:0/1005566520 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4cbc09e760 con 0x7f4cc4107960 2024-04-25T20:58:34.612 INFO:teuthology.orchestra.run.smithi047.stdout:77309411408 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cb40a3cd0 msgr2=0x7f4cb40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cb40a3cd0 0x7f4cb40a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005950 tx=0x7f4cb800b410 comp rx=0 tx=0).stop 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4107960 msgr2=0x7f4cc40ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4107960 0x7f4cc40ff410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00b830 tx=0x7f4cbc004ce0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 shutdown_connections 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4cb40a3cd0 0x7f4cb40a6190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc4107960 0x7f4cc40ff410 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc4101960 0x7f4cc40fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 --2- 172.21.15.47:0/1005566520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cc4102320 0x7f4cc40feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 >> 172.21.15.47:0/1005566520 conn(0x7f4cc40789e0 msgr2=0x7f4cc406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 shutdown_connections 2024-04-25T20:58:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:34.614+0000 7f4ccacda700 1 -- 172.21.15.47:0/1005566520 wait complete. 2024-04-25T20:58:34.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:34 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3162756793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T20:58:34.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:34 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1005566520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T20:58:35.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:34 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3162756793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T20:58:35.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:34 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1005566520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T20:58:35.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:34 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3162756793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T20:58:35.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:34 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1005566520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T20:58:35.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:35 smithi047 ceph-mon[29195]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:35 smithi186 ceph-mon[32529]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 -- 172.21.15.47:0/3830601533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 msgr2=0x7fe8cc111890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 --2- 172.21.15.47:0/3830601533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 0x7fe8cc111890 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400b0a0 tx=0x7fe8c400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 -- 172.21.15.47:0/3830601533 shutdown_connections 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 --2- 172.21.15.47:0/3830601533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 0x7fe8cc111890 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 --2- 172.21.15.47:0/3830601533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8cc074c70 0x7fe8cc075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 --2- 172.21.15.47:0/3830601533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc075590 0x7fe8cc10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 -- 172.21.15.47:0/3830601533 >> 172.21.15.47:0/3830601533 conn(0x7fe8cc079ba0 msgr2=0x7fe8cc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 -- 172.21.15.47:0/3830601533 shutdown_connections 2024-04-25T20:58:35.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 -- 172.21.15.47:0/3830601533 wait complete. 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 Processor -- start 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.990+0000 7fe8d4508700 1 -- start start 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d4508700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc074c70 0x7fe8cc19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d4508700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8cc075590 0x7fe8cc19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d4508700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 0x7fe8cc19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d4508700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc114e30 con 0x7fe8cc10f4a0 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d4508700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc114cb0 con 0x7fe8cc075590 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d4508700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc114fb0 con 0x7fe8cc074c70 2024-04-25T20:58:35.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d2aa5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 0x7fe8cc19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d1aa3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8cc075590 0x7fe8cc19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d22a4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc074c70 0x7fe8cc19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d2aa5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 0x7fe8cc19fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55028/0 (socket says 172.21.15.47:55028) 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d2aa5700 1 -- 172.21.15.47:0/2945750157 learned_addr learned my addr 172.21.15.47:0/2945750157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d22a4700 1 -- 172.21.15.47:0/2945750157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8cc075590 msgr2=0x7fe8cc19b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d22a4700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8cc075590 0x7fe8cc19b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d22a4700 1 -- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 msgr2=0x7fe8cc19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d22a4700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 0x7fe8cc19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d22a4700 1 -- 172.21.15.47:0/2945750157 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc1a0250 con 0x7fe8cc074c70 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d2aa5700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 0x7fe8cc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.991+0000 7fe8d1aa3700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8cc075590 0x7fe8cc19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.992+0000 7fe8d22a4700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc074c70 0x7fe8cc19b240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00d750 tx=0x7fe8bc00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.992+0000 7fe8cb7fe700 1 -- 172.21.15.47:0/2945750157 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc014030 con 0x7fe8cc074c70 2024-04-25T20:58:35.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.992+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8cc1a0540 con 0x7fe8cc074c70 2024-04-25T20:58:35.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.992+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8cc1a7dc0 con 0x7fe8cc074c70 2024-04-25T20:58:35.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.992+0000 7fe8cb7fe700 1 -- 172.21.15.47:0/2945750157 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8bc0044d0 con 0x7fe8cc074c70 2024-04-25T20:58:35.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.992+0000 7fe8cb7fe700 1 -- 172.21.15.47:0/2945750157 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc01d060 con 0x7fe8cc074c70 2024-04-25T20:58:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.993+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8cc069730 con 0x7fe8cc074c70 2024-04-25T20:58:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.994+0000 7fe8cb7fe700 1 -- 172.21.15.47:0/2945750157 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe8bc0049e0 con 0x7fe8cc074c70 2024-04-25T20:58:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.995+0000 7fe8cb7fe700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe8b40a3cd0 0x7fe8b40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.995+0000 7fe8d1aa3700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe8b40a3cd0 0x7fe8b40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.995+0000 7fe8cb7fe700 1 -- 172.21.15.47:0/2945750157 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fe8bc0d02b0 con 0x7fe8cc074c70 2024-04-25T20:58:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.995+0000 7fe8d1aa3700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe8b40a3cd0 0x7fe8b40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc110f80 tx=0x7fe8c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.999+0000 7fe8cb7fe700 1 -- 172.21.15.47:0/2945750157 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8bc004750 con 0x7fe8cc074c70 2024-04-25T20:58:36.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:35 smithi092 ceph-mon[32524]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:36.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.211+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe8cc1a7f90 con 0x7fe8cc074c70 2024-04-25T20:58:36.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.211+0000 7fe8cb7fe700 1 -- 172.21.15.47:0/2945750157 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe8bc09cac0 con 0x7fe8cc074c70 2024-04-25T20:58:36.212 INFO:teuthology.orchestra.run.smithi047.stdout:55834574932 2024-04-25T20:58:36.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.213+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe8b40a3cd0 msgr2=0x7fe8b40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.213+0000 7fe8d4508700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe8b40a3cd0 0x7fe8b40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc110f80 tx=0x7fe8c000b410 comp rx=0 tx=0).stop 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.213+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc074c70 msgr2=0x7fe8cc19b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.213+0000 7fe8d4508700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc074c70 0x7fe8cc19b240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00d750 tx=0x7fe8bc00da60 comp rx=0 tx=0).stop 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.214+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 shutdown_connections 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.214+0000 7fe8d4508700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fe8b40a3cd0 0x7fe8b40a6190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.214+0000 7fe8d4508700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8cc10f4a0 0x7fe8cc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.214+0000 7fe8d4508700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc074c70 0x7fe8cc19b240 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.214+0000 7fe8d4508700 1 --2- 172.21.15.47:0/2945750157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8cc075590 0x7fe8cc19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.214+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 >> 172.21.15.47:0/2945750157 conn(0x7fe8cc079ba0 msgr2=0x7fe8cc100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.214+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 shutdown_connections 2024-04-25T20:58:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.214+0000 7fe8d4508700 1 -- 172.21.15.47:0/2945750157 wait complete. 2024-04-25T20:58:36.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724606 got 167503724610 for osd.5 2024-04-25T20:58:36.533 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:36.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.882+0000 7f59c49a7700 1 -- 172.21.15.47:0/614426702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 msgr2=0x7f59c0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.882+0000 7f59c49a7700 1 --2- 172.21.15.47:0/614426702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 0x7f59c0101490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009990 tx=0x7f59b4009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 -- 172.21.15.47:0/614426702 shutdown_connections 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 --2- 172.21.15.47:0/614426702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c01019d0 0x7f59c010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 --2- 172.21.15.47:0/614426702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59c0108ad0 0x7f59c0108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 --2- 172.21.15.47:0/614426702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 0x7f59c0101490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 -- 172.21.15.47:0/614426702 >> 172.21.15.47:0/614426702 conn(0x7f59c0077fb0 msgr2=0x7f59c00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 -- 172.21.15.47:0/614426702 shutdown_connections 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 -- 172.21.15.47:0/614426702 wait complete. 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 Processor -- start 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.883+0000 7f59c49a7700 1 -- start start 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59c49a7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 0x7f59c0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59c49a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59c01019d0 0x7f59c01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:36.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59c49a7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c0108ad0 0x7f59c019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59c49a7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0112c20 con 0x7f59c0101030 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59c49a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0112aa0 con 0x7f59c01019d0 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 0x7f59c0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59c49a7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0112da0 con 0x7f59c0108ad0 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 0x7f59c0199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55018/0 (socket says 172.21.15.47:55018) 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 -- 172.21.15.47:0/3428049062 learned_addr learned my addr 172.21.15.47:0/3428049062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59bdd9b700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59c01019d0 0x7f59c01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59bed9d700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c0108ad0 0x7f59c019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 -- 172.21.15.47:0/3428049062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c0108ad0 msgr2=0x7f59c019da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c0108ad0 0x7f59c019da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 -- 172.21.15.47:0/3428049062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59c01019d0 msgr2=0x7f59c01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59c01019d0 0x7f59c01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 -- 172.21.15.47:0/3428049062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c019e180 con 0x7f59c0101030 2024-04-25T20:58:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59be59c700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 0x7f59c0199170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f59b000d980 tx=0x7f59b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.884+0000 7f59af7fe700 1 -- 172.21.15.47:0/3428049062 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0016070 con 0x7f59c0101030 2024-04-25T20:58:36.625 INFO:teuthology.orchestra.run.smithi047.stdout:292057776174 2024-04-25T20:58:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.885+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c019e470 con 0x7f59c0101030 2024-04-25T20:58:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.885+0000 7f59af7fe700 1 -- 172.21.15.47:0/3428049062 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b0004830 con 0x7f59c0101030 2024-04-25T20:58:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.885+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c01a5cf0 con 0x7f59c0101030 2024-04-25T20:58:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.885+0000 7f59af7fe700 1 -- 172.21.15.47:0/3428049062 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0005540 con 0x7f59c0101030 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.885+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c0104350 con 0x7f59c0101030 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.887+0000 7f59af7fe700 1 -- 172.21.15.47:0/3428049062 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f59b0004220 con 0x7f59c0101030 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.887+0000 7f59af7fe700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a80a3cd0 0x7f59a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.888+0000 7f59af7fe700 1 -- 172.21.15.47:0/3428049062 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f59b00d07f0 con 0x7f59c0101030 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.888+0000 7f59bdd9b700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a80a3cd0 0x7f59a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.888+0000 7f59bdd9b700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a80a3cd0 0x7f59a80a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b5c0 tx=0x7f59b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:35.895+0000 7f59af7fe700 1 -- 172.21.15.47:0/3428049062 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59b001f020 con 0x7f59c0101030 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.121+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f59c019a4d0 con 0x7f59c0101030 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.122+0000 7f59af7fe700 1 -- 172.21.15.47:0/3428049062 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f59b009d060 con 0x7f59c0101030 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a80a3cd0 msgr2=0x7f59a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a80a3cd0 0x7f59a80a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b5c0 tx=0x7f59b4005960 comp rx=0 tx=0).stop 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 msgr2=0x7f59c0199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 0x7f59c0199170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f59b000d980 tx=0x7f59b000dc90 comp rx=0 tx=0).stop 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 shutdown_connections 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f59a80a3cd0 0x7f59a80a6190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c0101030 0x7f59c0199170 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c0108ad0 0x7f59c019da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 --2- 172.21.15.47:0/3428049062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59c01019d0 0x7f59c01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 >> 172.21.15.47:0/3428049062 conn(0x7f59c0077fb0 msgr2=0x7f59c00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 shutdown_connections 2024-04-25T20:58:36.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:36.124+0000 7f59c49a7700 1 -- 172.21.15.47:0/3428049062 wait complete. 2024-04-25T20:58:36.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:36 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3428049062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T20:58:36.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:36 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2945750157' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T20:58:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:36 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3428049062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T20:58:36.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:36 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2945750157' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T20:58:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:36 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3428049062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T20:58:37.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:36 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2945750157' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T20:58:37.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 -- 172.21.15.47:0/2810083837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c103460 msgr2=0x7fdb9c10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:37.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 --2- 172.21.15.47:0/2810083837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c103460 0x7fdb9c10f940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400b0a0 tx=0x7fdb9400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 -- 172.21.15.47:0/2810083837 shutdown_connections 2024-04-25T20:58:37.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 --2- 172.21.15.47:0/2810083837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c103460 0x7fdb9c10f940 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 --2- 172.21.15.47:0/2810083837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c108a70 0x7fdb9c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 --2- 172.21.15.47:0/2810083837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c102ac0 0x7fdb9c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 -- 172.21.15.47:0/2810083837 >> 172.21.15.47:0/2810083837 conn(0x7fdb9c079bf0 msgr2=0x7fdb9c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 -- 172.21.15.47:0/2810083837 shutdown_connections 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.834+0000 7fdba1457700 1 -- 172.21.15.47:0/2810083837 wait complete. 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.835+0000 7fdba1457700 1 Processor -- start 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.835+0000 7fdba1457700 1 -- start start 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.835+0000 7fdba1457700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c102ac0 0x7fdb9c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.835+0000 7fdba1457700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c103460 0x7fdb9c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.835+0000 7fdba1457700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c108a70 0x7fdb9c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.835+0000 7fdba1457700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c112c10 con 0x7fdb9c102ac0 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.835+0000 7fdba1457700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c112a90 con 0x7fdb9c103460 2024-04-25T20:58:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.835+0000 7fdba1457700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c112d90 con 0x7fdb9c108a70 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c102ac0 0x7fdb9c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c102ac0 0x7fdb9c199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58790/0 (socket says 172.21.15.47:58790) 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 -- 172.21.15.47:0/1015686168 learned_addr learned my addr 172.21.15.47:0/1015686168 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9a7fc700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c103460 0x7fdb9c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9b7fe700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c108a70 0x7fdb9c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 -- 172.21.15.47:0/1015686168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c108a70 msgr2=0x7fdb9c19d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c108a70 0x7fdb9c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 -- 172.21.15.47:0/1015686168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c103460 msgr2=0x7fdb9c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c103460 0x7fdb9c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 -- 172.21.15.47:0/1015686168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c19e040 con 0x7fdb9c102ac0 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb9affd700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c102ac0 0x7fdb9c199080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00c9b0 tx=0x7fdb8c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb83fff700 1 -- 172.21.15.47:0/1015686168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c007b00 con 0x7fdb9c102ac0 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb83fff700 1 -- 172.21.15.47:0/1015686168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8c007ca0 con 0x7fdb9c102ac0 2024-04-25T20:58:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c19e330 con 0x7fdb9c102ac0 2024-04-25T20:58:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdb83fff700 1 -- 172.21.15.47:0/1015686168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c018730 con 0x7fdb9c102ac0 2024-04-25T20:58:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.836+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c1a5bb0 con 0x7fdb9c102ac0 2024-04-25T20:58:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.837+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c069730 con 0x7fdb9c102ac0 2024-04-25T20:58:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.838+0000 7fdb83fff700 1 -- 172.21.15.47:0/1015686168 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb8c00f450 con 0x7fdb9c102ac0 2024-04-25T20:58:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.839+0000 7fdb83fff700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdb840a3cd0 0x7fdb840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:37.381 INFO:teuthology.orchestra.run.smithi047.stdout:141733920838 2024-04-25T20:58:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.839+0000 7fdb9a7fc700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdb840a3cd0 0x7fdb840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.839+0000 7fdb83fff700 1 -- 172.21.15.47:0/1015686168 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fdb8c0d05f0 con 0x7fdb9c102ac0 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.839+0000 7fdb9a7fc700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdb840a3cd0 0x7fdb840a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005950 tx=0x7fdb900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:32.844+0000 7fdb83fff700 1 -- 172.21.15.47:0/1015686168 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb8c014030 con 0x7fdb9c102ac0 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.061+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdb9c02df80 con 0x7fdb9c102ac0 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.061+0000 7fdb83fff700 1 -- 172.21.15.47:0/1015686168 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb8c09ce00 con 0x7fdb9c102ac0 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdb840a3cd0 msgr2=0x7fdb840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdb840a3cd0 0x7fdb840a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005950 tx=0x7fdb900058e0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c102ac0 msgr2=0x7fdb9c199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c102ac0 0x7fdb9c199080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00c9b0 tx=0x7fdb8c00cd70 comp rx=0 tx=0).stop 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 shutdown_connections 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdb840a3cd0 0x7fdb840a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb9c102ac0 0x7fdb9c199080 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c108a70 0x7fdb9c19d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 --2- 172.21.15.47:0/1015686168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c103460 0x7fdb9c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:37.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 >> 172.21.15.47:0/1015686168 conn(0x7fdb9c079bf0 msgr2=0x7fdb9c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:37.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 shutdown_connections 2024-04-25T20:58:37.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:33.063+0000 7fdba1457700 1 -- 172.21.15.47:0/1015686168 wait complete. 2024-04-25T20:58:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:37 smithi186 ceph-mon[32529]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:37 smithi092 ceph-mon[32524]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:38.089 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:37 smithi047 ceph-mon[29195]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:38.124 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920834 got 141733920838 for osd.4 2024-04-25T20:58:38.125 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:38.513 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351006 got 347892351012 for osd.11 2024-04-25T20:58:38.514 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:38.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 -- 172.21.15.47:0/2513271659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c107990 msgr2=0x7f819c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:38.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 --2- 172.21.15.47:0/2513271659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c107990 0x7f819c107d70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f818c009990 tx=0x7f818c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 -- 172.21.15.47:0/2513271659 shutdown_connections 2024-04-25T20:58:38.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 --2- 172.21.15.47:0/2513271659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c107990 0x7f819c107d70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 --2- 172.21.15.47:0/2513271659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f819c102300 0x7f819c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 --2- 172.21.15.47:0/2513271659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f819c101940 0x7f819c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 -- 172.21.15.47:0/2513271659 >> 172.21.15.47:0/2513271659 conn(0x7f819c0788f0 msgr2=0x7f819c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 -- 172.21.15.47:0/2513271659 shutdown_connections 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 -- 172.21.15.47:0/2513271659 wait complete. 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.529+0000 7f81a1597700 1 Processor -- start 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f81a1597700 1 -- start start 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f81a1597700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f819c101940 0x7f819c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f81a1597700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c102300 0x7f819c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f81a1597700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f819c107990 0x7f819c19d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f81a1597700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f819c111ae0 con 0x7f819c102300 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f81a1597700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f819c111960 con 0x7f819c107990 2024-04-25T20:58:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f81a1597700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f819c111c60 con 0x7f819c101940 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c102300 0x7f819c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c102300 0x7f819c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55050/0 (socket says 172.21.15.47:55050) 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 -- 172.21.15.47:0/753084156 learned_addr learned my addr 172.21.15.47:0/753084156 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819b7fe700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f819c107990 0x7f819c19d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819affd700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f819c101940 0x7f819c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 -- 172.21.15.47:0/753084156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f819c101940 msgr2=0x7f819c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f819c101940 0x7f819c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 -- 172.21.15.47:0/753084156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f819c107990 msgr2=0x7f819c19d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f819c107990 0x7f819c19d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 -- 172.21.15.47:0/753084156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c19e100 con 0x7f819c102300 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819affd700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f819c101940 0x7f819c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819b7fe700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f819c107990 0x7f819c19d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.530+0000 7f819a7fc700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c102300 0x7f819c199630 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f819000c920 tx=0x7f819000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.531+0000 7f8183fff700 1 -- 172.21.15.47:0/753084156 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190007b00 con 0x7f819c102300 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.531+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c19e3f0 con 0x7f819c102300 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.531+0000 7f8183fff700 1 -- 172.21.15.47:0/753084156 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8190004ce0 con 0x7f819c102300 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.531+0000 7f8183fff700 1 -- 172.21.15.47:0/753084156 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190005650 con 0x7f819c102300 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.531+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1a5c70 con 0x7f819c102300 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.532+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c069730 con 0x7f819c102300 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.534+0000 7f8183fff700 1 -- 172.21.15.47:0/753084156 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8190004750 con 0x7f819c102300 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.535+0000 7f8183fff700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f81840a3cd0 0x7f81840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.535+0000 7f819affd700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f81840a3cd0 0x7f81840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.535+0000 7f8183fff700 1 -- 172.21.15.47:0/753084156 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f81900d0620 con 0x7f819c102300 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.535+0000 7f819affd700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f81840a3cd0 0x7f81840a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f818c006010 tx=0x7f818c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.538+0000 7f8183fff700 1 -- 172.21.15.47:0/753084156 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8190014030 con 0x7f819c102300 2024-04-25T20:58:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.751+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f819c1a5f00 con 0x7f819c102300 2024-04-25T20:58:38.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.752+0000 7f8183fff700 1 -- 172.21.15.47:0/753084156 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f819009ce30 con 0x7f819c102300 2024-04-25T20:58:38.753 INFO:teuthology.orchestra.run.smithi047.stdout:34359738459 2024-04-25T20:58:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f81840a3cd0 msgr2=0x7f81840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f81840a3cd0 0x7f81840a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f818c006010 tx=0x7f818c00b540 comp rx=0 tx=0).stop 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c102300 msgr2=0x7f819c199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c102300 0x7f819c199630 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f819000c920 tx=0x7f819000cce0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 shutdown_connections 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f81840a3cd0 0x7f81840a6190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f819c107990 0x7f819c19d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f819c101940 0x7f819c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 --2- 172.21.15.47:0/753084156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c102300 0x7f819c199630 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 >> 172.21.15.47:0/753084156 conn(0x7f819c0788f0 msgr2=0x7f819c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 shutdown_connections 2024-04-25T20:58:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:38.754+0000 7f81a1597700 1 -- 172.21.15.47:0/753084156 wait complete. 2024-04-25T20:58:38.997 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776170 got 292057776174 for osd.9 2024-04-25T20:58:38.997 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:39.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:39 smithi047 ceph-mon[29195]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:39.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:39 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/753084156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T20:58:39.952 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299443 got 223338299446 for osd.7 2024-04-25T20:58:39.952 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:39.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:39 smithi186 ceph-mon[32529]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:39.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:39 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/753084156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T20:58:40.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:39 smithi092 ceph-mon[32524]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:40.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:39 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/753084156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T20:58:40.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411408 for osd.2 2024-04-25T20:58:40.355 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:40.783 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574932 for osd.1 2024-04-25T20:58:40.783 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 -- 172.21.15.47:0/3777653496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 msgr2=0x7fef3410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/3777653496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 0x7fef3410f900 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009990 tx=0x7fef2c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 -- 172.21.15.47:0/3777653496 shutdown_connections 2024-04-25T20:58:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/3777653496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 0x7fef3410f900 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/3777653496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef34108a30 0x7fef34108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/3777653496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef3406a750 0x7fef3406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 -- 172.21.15.47:0/3777653496 >> 172.21.15.47:0/3777653496 conn(0x7fef340788f0 msgr2=0x7fef34078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 -- 172.21.15.47:0/3777653496 shutdown_connections 2024-04-25T20:58:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.991+0000 7fef3a1a9700 1 -- 172.21.15.47:0/3777653496 wait complete. 2024-04-25T20:58:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef3a1a9700 1 Processor -- start 2024-04-25T20:58:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef3a1a9700 1 -- start start 2024-04-25T20:58:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef3a1a9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef3406a750 0x7fef34199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef3a1a9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 0x7fef341995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef3a1a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef34108a30 0x7fef3419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef3a1a9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef34112c10 con 0x7fef3406b0f0 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef3a1a9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef34112a90 con 0x7fef34108a30 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef3a1a9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fef34112d90 con 0x7fef3406a750 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef32ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 0x7fef341995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef32ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 0x7fef341995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55070/0 (socket says 172.21.15.47:55070) 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.992+0000 7fef32ffd700 1 -- 172.21.15.47:0/1545023644 learned_addr learned my addr 172.21.15.47:0/1545023644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef337fe700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef3406a750 0x7fef34199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef33fff700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef34108a30 0x7fef3419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef32ffd700 1 -- 172.21.15.47:0/1545023644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef3406a750 msgr2=0x7fef34199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef32ffd700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef3406a750 0x7fef34199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef32ffd700 1 -- 172.21.15.47:0/1545023644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef34108a30 msgr2=0x7fef3419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef32ffd700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef34108a30 0x7fef3419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef32ffd700 1 -- 172.21.15.47:0/1545023644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3419e090 con 0x7fef3406b0f0 2024-04-25T20:58:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef32ffd700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 0x7fef341995c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef2800c970 tx=0x7fef2800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef30ff9700 1 -- 172.21.15.47:0/1545023644 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28007aa0 con 0x7fef3406b0f0 2024-04-25T20:58:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef30ff9700 1 -- 172.21.15.47:0/1545023644 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef28004ce0 con 0x7fef3406b0f0 2024-04-25T20:58:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3419e380 con 0x7fef3406b0f0 2024-04-25T20:58:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef30ff9700 1 -- 172.21.15.47:0/1545023644 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28005510 con 0x7fef3406b0f0 2024-04-25T20:58:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.993+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef341a5c00 con 0x7fef3406b0f0 2024-04-25T20:58:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.994+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef34069730 con 0x7fef3406b0f0 2024-04-25T20:58:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.995+0000 7fef30ff9700 1 -- 172.21.15.47:0/1545023644 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fef28007c40 con 0x7fef3406b0f0 2024-04-25T20:58:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.996+0000 7fef30ff9700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fef1c0a3cd0 0x7fef1c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.996+0000 7fef337fe700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fef1c0a3cd0 0x7fef1c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.997+0000 7fef30ff9700 1 -- 172.21.15.47:0/1545023644 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fef280d0350 con 0x7fef3406b0f0 2024-04-25T20:58:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:40.997+0000 7fef337fe700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fef1c0a3cd0 0x7fef1c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef24007fd0 tx=0x7fef240079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:41.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.001+0000 7fef30ff9700 1 -- 172.21.15.47:0/1545023644 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef2809cc50 con 0x7fef3406b0f0 2024-04-25T20:58:41.118 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:41.230 INFO:teuthology.orchestra.run.smithi047.stdout:107374182476 2024-04-25T20:58:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.229+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fef3402df80 con 0x7fef3406b0f0 2024-04-25T20:58:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.229+0000 7fef30ff9700 1 -- 172.21.15.47:0/1545023644 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fef2809c550 con 0x7fef3406b0f0 2024-04-25T20:58:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fef1c0a3cd0 msgr2=0x7fef1c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fef1c0a3cd0 0x7fef1c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef24007fd0 tx=0x7fef240079e0 comp rx=0 tx=0).stop 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 msgr2=0x7fef341995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 0x7fef341995c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef2800c970 tx=0x7fef2800cc80 comp rx=0 tx=0).stop 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 shutdown_connections 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fef1c0a3cd0 0x7fef1c0a6190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef34108a30 0x7fef3419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef3406a750 0x7fef34199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 --2- 172.21.15.47:0/1545023644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef3406b0f0 0x7fef341995c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 >> 172.21.15.47:0/1545023644 conn(0x7fef340788f0 msgr2=0x7fef340ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 shutdown_connections 2024-04-25T20:58:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.231+0000 7fef3a1a9700 1 -- 172.21.15.47:0/1545023644 wait complete. 2024-04-25T20:58:41.641 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738459 for osd.0 2024-04-25T20:58:41.641 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:41.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:41 smithi186 ceph-mon[32529]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:41.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:41 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1545023644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T20:58:42.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:41 smithi092 ceph-mon[32524]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:42.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:41 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1545023644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T20:58:42.096 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:41 smithi047 ceph-mon[29195]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:42.096 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:41 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1545023644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T20:58:42.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182476 for osd.3 2024-04-25T20:58:42.512 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 -- 172.21.15.47:0/4137791890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 msgr2=0x7f86bc10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 --2- 172.21.15.47:0/4137791890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 0x7f86bc10f8b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86b400b0a0 tx=0x7f86b400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 -- 172.21.15.47:0/4137791890 shutdown_connections 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 --2- 172.21.15.47:0/4137791890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 0x7f86bc10f8b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 --2- 172.21.15.47:0/4137791890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86bc108a50 0x7f86bc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 --2- 172.21.15.47:0/4137791890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc102a90 0x7f86bc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 -- 172.21.15.47:0/4137791890 >> 172.21.15.47:0/4137791890 conn(0x7f86bc079ba0 msgr2=0x7f86bc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 -- 172.21.15.47:0/4137791890 shutdown_connections 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.876+0000 7f86c20ac700 1 -- 172.21.15.47:0/4137791890 wait complete. 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86c20ac700 1 Processor -- start 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86c20ac700 1 -- start start 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86c20ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc102a90 0x7f86bc199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86c20ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 0x7f86bc199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86c20ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86bc108a50 0x7f86bc19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86c20ac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc112c60 con 0x7f86bc103450 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86c20ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc112ae0 con 0x7f86bc102a90 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86c20ac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc112de0 con 0x7f86bc108a50 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86baffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 0x7f86bc199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:42.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86baffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 0x7f86bc199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55088/0 (socket says 172.21.15.47:55088) 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86baffd700 1 -- 172.21.15.47:0/423106243 learned_addr learned my addr 172.21.15.47:0/423106243 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86bb7fe700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc102a90 0x7f86bc199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.877+0000 7f86bbfff700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86bc108a50 0x7f86bc19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86baffd700 1 -- 172.21.15.47:0/423106243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86bc108a50 msgr2=0x7f86bc19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86baffd700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86bc108a50 0x7f86bc19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86baffd700 1 -- 172.21.15.47:0/423106243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc102a90 msgr2=0x7f86bc199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86baffd700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc102a90 0x7f86bc199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86baffd700 1 -- 172.21.15.47:0/423106243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc19e140 con 0x7f86bc103450 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86baffd700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 0x7f86bc199670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86b000d980 tx=0x7f86b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86b8ff9700 1 -- 172.21.15.47:0/423106243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0016070 con 0x7f86bc103450 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86b8ff9700 1 -- 172.21.15.47:0/423106243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b0004830 con 0x7f86bc103450 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86b8ff9700 1 -- 172.21.15.47:0/423106243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0005540 con 0x7f86bc103450 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc19e430 con 0x7f86bc103450 2024-04-25T20:58:42.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.878+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1a5cb0 con 0x7f86bc103450 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.879+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc069730 con 0x7f86bc103450 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.880+0000 7f86b8ff9700 1 -- 172.21.15.47:0/423106243 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86b0004220 con 0x7f86bc103450 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.881+0000 7f86b8ff9700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f86a40a3cd0 0x7f86a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.881+0000 7f86bb7fe700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f86a40a3cd0 0x7f86a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.881+0000 7f86b8ff9700 1 -- 172.21.15.47:0/423106243 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f86b00d08c0 con 0x7f86bc103450 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.881+0000 7f86bb7fe700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f86a40a3cd0 0x7f86a40a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009a10 tx=0x7f86ac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:41.884+0000 7f86b8ff9700 1 -- 172.21.15.47:0/423106243 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86b001f020 con 0x7f86bc103450 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stdout:257698037813 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.095+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f86bc02df80 con 0x7f86bc103450 2024-04-25T20:58:42.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.095+0000 7f86b8ff9700 1 -- 172.21.15.47:0/423106243 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f86b009d0d0 con 0x7f86bc103450 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.097+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f86a40a3cd0 msgr2=0x7f86a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.097+0000 7f86c20ac700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f86a40a3cd0 0x7f86a40a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009a10 tx=0x7f86ac009190 comp rx=0 tx=0).stop 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.097+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 msgr2=0x7f86bc199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.097+0000 7f86c20ac700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 0x7f86bc199670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86b000d980 tx=0x7f86b000dc90 comp rx=0 tx=0).stop 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.098+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 shutdown_connections 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.098+0000 7f86c20ac700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f86a40a3cd0 0x7f86a40a6190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.098+0000 7f86c20ac700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86bc108a50 0x7f86bc19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.098+0000 7f86c20ac700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc102a90 0x7f86bc199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.098+0000 7f86c20ac700 1 --2- 172.21.15.47:0/423106243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86bc103450 0x7f86bc199670 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.098+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 >> 172.21.15.47:0/423106243 conn(0x7f86bc079ba0 msgr2=0x7f86bc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.098+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 shutdown_connections 2024-04-25T20:58:42.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:42.098+0000 7f86c20ac700 1 -- 172.21.15.47:0/423106243 wait complete. 2024-04-25T20:58:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:42 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/423106243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T20:58:42.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:42 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/423106243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T20:58:43.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:42 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/423106243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T20:58:43.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037808 got 257698037813 for osd.8 2024-04-25T20:58:43.315 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:43.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:43 smithi047 ceph-mon[29195]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:43.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:43 smithi186 ceph-mon[32529]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:44.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:43 smithi092 ceph-mon[32524]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:58:45.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 -- 172.21.15.47:0/3787298977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d4406bb30 msgr2=0x7f5d4410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 --2- 172.21.15.47:0/3787298977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d4406bb30 0x7f5d4410f900 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400b0a0 tx=0x7f5d3400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 -- 172.21.15.47:0/3787298977 shutdown_connections 2024-04-25T20:58:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 --2- 172.21.15.47:0/3787298977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d4406bb30 0x7f5d4410f900 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 --2- 172.21.15.47:0/3787298977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d44108a30 0x7f5d44108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 --2- 172.21.15.47:0/3787298977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4406b190 0x7f5d4406b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 -- 172.21.15.47:0/3787298977 >> 172.21.15.47:0/3787298977 conn(0x7f5d44078910 msgr2=0x7f5d44078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 -- 172.21.15.47:0/3787298977 shutdown_connections 2024-04-25T20:58:45.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 -- 172.21.15.47:0/3787298977 wait complete. 2024-04-25T20:58:45.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 Processor -- start 2024-04-25T20:58:45.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.089+0000 7f5d4c184700 1 -- start start 2024-04-25T20:58:45.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4c184700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4406b190 0x7f5d44101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4c184700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4406bb30 0x7f5d44101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4c184700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d44108a30 0x7f5d441024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4c184700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44112bd0 con 0x7f5d44108a30 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4c184700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44112a50 con 0x7f5d4406bb30 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4c184700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44112d50 con 0x7f5d4406b190 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d44108a30 0x7f5d441024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d44108a30 0x7f5d441024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48008/0 (socket says 172.21.15.47:48008) 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 -- 172.21.15.47:0/1357044109 learned_addr learned my addr 172.21.15.47:0/1357044109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d49f20700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4406b190 0x7f5d44101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:45.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4971f700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4406bb30 0x7f5d44101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 -- 172.21.15.47:0/1357044109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4406b190 msgr2=0x7f5d44101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4406b190 0x7f5d44101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 -- 172.21.15.47:0/1357044109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4406bb30 msgr2=0x7f5d44101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4406bb30 0x7f5d44101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 -- 172.21.15.47:0/1357044109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d441a7570 con 0x7f5d44108a30 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4a721700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d44108a30 0x7f5d441024c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400b830 tx=0x7f5d34004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d42ffd700 1 -- 172.21.15.47:0/1357044109 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34021070 con 0x7f5d44108a30 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d42ffd700 1 -- 172.21.15.47:0/1357044109 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d34007e30 con 0x7f5d44108a30 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d441a7800 con 0x7f5d44108a30 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.090+0000 7f5d42ffd700 1 -- 172.21.15.47:0/1357044109 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340047d0 con 0x7f5d44108a30 2024-04-25T20:58:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.091+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d441a7d00 con 0x7f5d44108a30 2024-04-25T20:58:45.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.092+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d44069730 con 0x7f5d44108a30 2024-04-25T20:58:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.093+0000 7f5d42ffd700 1 -- 172.21.15.47:0/1357044109 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d34007980 con 0x7f5d44108a30 2024-04-25T20:58:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.093+0000 7f5d42ffd700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5d2c0a3d50 0x7f5d2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.094+0000 7f5d49f20700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5d2c0a3d50 0x7f5d2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.094+0000 7f5d42ffd700 1 -- 172.21.15.47:0/1357044109 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f5d3400f020 con 0x7f5d44108a30 2024-04-25T20:58:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.094+0000 7f5d49f20700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5d2c0a3d50 0x7f5d2c0a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005950 tx=0x7f5d3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.097+0000 7f5d42ffd700 1 -- 172.21.15.47:0/1357044109 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d3402a020 con 0x7f5d44108a30 2024-04-25T20:58:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.307+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5d4402df80 con 0x7f5d44108a30 2024-04-25T20:58:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.307+0000 7f5d42ffd700 1 -- 172.21.15.47:0/1357044109 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5d3409e7a0 con 0x7f5d44108a30 2024-04-25T20:58:45.308 INFO:teuthology.orchestra.run.smithi047.stdout:317827579948 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5d2c0a3d50 msgr2=0x7f5d2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5d2c0a3d50 0x7f5d2c0a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005950 tx=0x7f5d3c00b410 comp rx=0 tx=0).stop 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d44108a30 msgr2=0x7f5d441024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d44108a30 0x7f5d441024c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400b830 tx=0x7f5d34004ce0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 shutdown_connections 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5d2c0a3d50 0x7f5d2c0a6210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d44108a30 0x7f5d441024c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4406b190 0x7f5d44101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 --2- 172.21.15.47:0/1357044109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4406bb30 0x7f5d44101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:45.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.309+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 >> 172.21.15.47:0/1357044109 conn(0x7f5d44078910 msgr2=0x7f5d440ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:45.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.310+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 shutdown_connections 2024-04-25T20:58:45.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:45.310+0000 7f5d4c184700 1 -- 172.21.15.47:0/1357044109 wait complete. 2024-04-25T20:58:45.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:45 smithi047 ceph-mon[29195]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:45.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:45 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1357044109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T20:58:45.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579942 got 317827579948 for osd.10 2024-04-25T20:58:45.959 DEBUG:teuthology.parallel:result is None 2024-04-25T20:58:45.959 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T20:58:45.959 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:58:45.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:45 smithi186 ceph-mon[32529]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:45.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:45 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1357044109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T20:58:46.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:45 smithi092 ceph-mon[32524]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:58:46.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:45 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1357044109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T20:58:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 -- 172.21.15.47:0/3526732532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234108a70 msgr2=0x7f5234108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 --2- 172.21.15.47:0/3526732532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234108a70 0x7f5234108e50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5224009990 tx=0x7f5224009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 -- 172.21.15.47:0/3526732532 shutdown_connections 2024-04-25T20:58:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 --2- 172.21.15.47:0/3526732532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234108a70 0x7f5234108e50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 --2- 172.21.15.47:0/3526732532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5234103480 0x7f523410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 --2- 172.21.15.47:0/3526732532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5234102ac0 0x7f5234102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 -- 172.21.15.47:0/3526732532 >> 172.21.15.47:0/3526732532 conn(0x7f5234079bb0 msgr2=0x7f5234079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 -- 172.21.15.47:0/3526732532 shutdown_connections 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.726+0000 7f52399f8700 1 -- 172.21.15.47:0/3526732532 wait complete. 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52399f8700 1 Processor -- start 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52399f8700 1 -- start start 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52399f8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234102ac0 0x7f52341992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52399f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5234103480 0x7f5234199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52399f8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5234108a70 0x7f523419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52399f8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5234112d00 con 0x7f5234102ac0 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52399f8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5234112b80 con 0x7f5234103480 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52399f8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5234112e80 con 0x7f5234108a70 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f5232ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234102ac0 0x7f52341992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f5232ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234102ac0 0x7f52341992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48020/0 (socket says 172.21.15.47:48020) 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f5232ffd700 1 -- 172.21.15.47:0/3866454728 learned_addr learned my addr 172.21.15.47:0/3866454728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52337fe700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5234108a70 0x7f523419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.727+0000 7f52327fc700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5234103480 0x7f5234199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f5232ffd700 1 -- 172.21.15.47:0/3866454728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5234108a70 msgr2=0x7f523419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f5232ffd700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5234108a70 0x7f523419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f5232ffd700 1 -- 172.21.15.47:0/3866454728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5234103480 msgr2=0x7f5234199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f5232ffd700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5234103480 0x7f5234199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f5232ffd700 1 -- 172.21.15.47:0/3866454728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523419e2e0 con 0x7f5234102ac0 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f5232ffd700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234102ac0 0x7f52341992d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5224000c00 tx=0x7f52240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f52389f6700 1 -- 172.21.15.47:0/3866454728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224027070 con 0x7f5234102ac0 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523419e570 con 0x7f5234102ac0 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f52389f6700 1 -- 172.21.15.47:0/3866454728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5224003fe0 con 0x7f5234102ac0 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341a5e50 con 0x7f5234102ac0 2024-04-25T20:58:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.728+0000 7f52389f6700 1 -- 172.21.15.47:0/3866454728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224022e30 con 0x7f5234102ac0 2024-04-25T20:58:47.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.729+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5234069730 con 0x7f5234102ac0 2024-04-25T20:58:47.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.730+0000 7f52389f6700 1 -- 172.21.15.47:0/3866454728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f522402a070 con 0x7f5234102ac0 2024-04-25T20:58:47.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.731+0000 7f52389f6700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f521c0a3c40 0x7f521c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.731+0000 7f52327fc700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f521c0a3c40 0x7f521c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.731+0000 7f52389f6700 1 -- 172.21.15.47:0/3866454728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f52240d13b0 con 0x7f5234102ac0 2024-04-25T20:58:47.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.731+0000 7f52327fc700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f521c0a3c40 0x7f521c0a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f523407a2e0 tx=0x7f5228009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.735+0000 7f52389f6700 1 -- 172.21.15.47:0/3866454728 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5224013070 con 0x7f5234102ac0 2024-04-25T20:58:47.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:47 smithi047 ceph-mon[29195]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T20:58:47.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:47 smithi047 ceph-mon[29195]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T20:58:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.938+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f523419a630 con 0x7f521c0a3c40 2024-04-25T20:58:47.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.946+0000 7f52389f6700 1 -- 172.21.15.47:0/3866454728 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258745 (secure 0 0 0) 0x7f523419a630 con 0x7f521c0a3c40 2024-04-25T20:58:47.950 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:58:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f521c0a3c40 msgr2=0x7f521c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f521c0a3c40 0x7f521c0a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f523407a2e0 tx=0x7f5228009190 comp rx=0 tx=0).stop 2024-04-25T20:58:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234102ac0 msgr2=0x7f52341992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234102ac0 0x7f52341992d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5224000c00 tx=0x7f52240047b0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 shutdown_connections 2024-04-25T20:58:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f521c0a3c40 0x7f521c0a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5234102ac0 0x7f52341992d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5234108a70 0x7f523419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 --2- 172.21.15.47:0/3866454728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5234103480 0x7f5234199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 >> 172.21.15.47:0/3866454728 conn(0x7f5234079bb0 msgr2=0x7f5234101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 shutdown_connections 2024-04-25T20:58:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:47.949+0000 7f52399f8700 1 -- 172.21.15.47:0/3866454728 wait complete. 2024-04-25T20:58:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-25T20:58:47.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:47 smithi186 ceph-mon[32529]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T20:58:47.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:47 smithi186 ceph-mon[32529]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T20:58:48.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:47 smithi092 ceph-mon[32524]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T20:58:48.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:47 smithi092 ceph-mon[32524]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T20:58:48.612 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-04-25T20:58:45.939815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655341,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":744,"num_read_kb":494,"num_write":349,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":324,"ondisk_log_size":324,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363064,"kb_used_data":13836,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578760,"statfs":{"total":1151940427776,"available":1151568650240,"internally_reserved":0,"allocated":14168064,"data_stored":7769907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002944"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.528159+0000","last_change":"2024-04-25T20:56:51.283722+0000","last_active":"2024-04-25T20:57:47.528159+0000","last_peered":"2024-04-25T20:57:47.528159+0000","last_clean":"2024-04-25T20:57:47.528159+0000","last_became_active":"2024-04-25T20:56:51.283359+0000","last_became_peered":"2024-04-25T20:56:51.283359+0000","last_unstale":"2024-04-25T20:57:47.528159+0000","last_undegraded":"2024-04-25T20:57:47.528159+0000","last_fullsized":"2024-04-25T20:57:47.528159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:47.314687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527554+0000","last_change":"2024-04-25T20:56:51.273199+0000","last_active":"2024-04-25T20:57:47.527554+0000","last_peered":"2024-04-25T20:57:47.527554+0000","last_clean":"2024-04-25T20:57:47.527554+0000","last_became_active":"2024-04-25T20:56:51.272838+0000","last_became_peered":"2024-04-25T20:56:51.272838+0000","last_unstale":"2024-04-25T20:57:47.527554+0000","last_undegraded":"2024-04-25T20:57:47.527554+0000","last_fullsized":"2024-04-25T20:57:47.527554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:58.033315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526369+0000","last_change":"2024-04-25T20:56:51.281461+0000","last_active":"2024-04-25T20:57:47.526369+0000","last_peered":"2024-04-25T20:57:47.526369+0000","last_clean":"2024-04-25T20:57:47.526369+0000","last_became_active":"2024-04-25T20:56:51.281141+0000","last_became_peered":"2024-04-25T20:56:51.281141+0000","last_unstale":"2024-04-25T20:57:47.526369+0000","last_undegraded":"2024-04-25T20:57:47.526369+0000","last_fullsized":"2024-04-25T20:57:47.526369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524353+0000","last_change":"2024-04-25T20:56:51.283187+0000","last_active":"2024-04-25T20:57:47.524353+0000","last_peered":"2024-04-25T20:57:47.524353+0000","last_clean":"2024-04-25T20:57:47.524353+0000","last_became_active":"2024-04-25T20:56:51.282640+0000","last_became_peered":"2024-04-25T20:56:51.282640+0000","last_unstale":"2024-04-25T20:57:47.524353+0000","last_undegraded":"2024-04-25T20:57:47.524353+0000","last_fullsized":"2024-04-25T20:57:47.524353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:15.588694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526553+0000","last_change":"2024-04-25T20:56:51.273204+0000","last_active":"2024-04-25T20:57:47.526553+0000","last_peered":"2024-04-25T20:57:47.526553+0000","last_clean":"2024-04-25T20:57:47.526553+0000","last_became_active":"2024-04-25T20:56:51.272126+0000","last_became_peered":"2024-04-25T20:56:51.272126+0000","last_unstale":"2024-04-25T20:57:47.526553+0000","last_undegraded":"2024-04-25T20:57:47.526553+0000","last_fullsized":"2024-04-25T20:57:47.526553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:31.684294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793774+0000","last_change":"2024-04-25T20:56:51.270762+0000","last_active":"2024-04-25T20:57:47.793774+0000","last_peered":"2024-04-25T20:57:47.793774+0000","last_clean":"2024-04-25T20:57:47.793774+0000","last_became_active":"2024-04-25T20:56:51.270444+0000","last_became_peered":"2024-04-25T20:56:51.270444+0000","last_unstale":"2024-04-25T20:57:47.793774+0000","last_undegraded":"2024-04-25T20:57:47.793774+0000","last_fullsized":"2024-04-25T20:57:47.793774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:00.155417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.530824+0000","last_change":"2024-04-25T20:56:51.276029+0000","last_active":"2024-04-25T20:57:47.530824+0000","last_peered":"2024-04-25T20:57:47.530824+0000","last_clean":"2024-04-25T20:57:47.530824+0000","last_became_active":"2024-04-25T20:56:51.275272+0000","last_became_peered":"2024-04-25T20:56:51.275272+0000","last_unstale":"2024-04-25T20:57:47.530824+0000","last_undegraded":"2024-04-25T20:57:47.530824+0000","last_fullsized":"2024-04-25T20:57:47.530824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.428444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.525409+0000","last_change":"2024-04-25T20:56:51.273769+0000","last_active":"2024-04-25T20:57:47.525409+0000","last_peered":"2024-04-25T20:57:47.525409+0000","last_clean":"2024-04-25T20:57:47.525409+0000","last_became_active":"2024-04-25T20:56:51.273412+0000","last_became_peered":"2024-04-25T20:56:51.273412+0000","last_unstale":"2024-04-25T20:57:47.525409+0000","last_undegraded":"2024-04-25T20:57:47.525409+0000","last_fullsized":"2024-04-25T20:57:47.525409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.324332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299708+0000","last_change":"2024-04-25T20:56:51.277097+0000","last_active":"2024-04-25T20:57:48.299708+0000","last_peered":"2024-04-25T20:57:48.299708+0000","last_clean":"2024-04-25T20:57:48.299708+0000","last_became_active":"2024-04-25T20:56:51.276768+0000","last_became_peered":"2024-04-25T20:56:51.276768+0000","last_unstale":"2024-04-25T20:57:48.299708+0000","last_undegraded":"2024-04-25T20:57:48.299708+0000","last_fullsized":"2024-04-25T20:57:48.299708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:23.276470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.794116+0000","last_change":"2024-04-25T20:56:51.279046+0000","last_active":"2024-04-25T20:57:47.794116+0000","last_peered":"2024-04-25T20:57:47.794116+0000","last_clean":"2024-04-25T20:57:47.794116+0000","last_became_active":"2024-04-25T20:56:51.278701+0000","last_became_peered":"2024-04-25T20:56:51.278701+0000","last_unstale":"2024-04-25T20:57:47.794116+0000","last_undegraded":"2024-04-25T20:57:47.794116+0000","last_fullsized":"2024-04-25T20:57:47.794116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.370482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531219+0000","last_change":"2024-04-25T20:56:51.271593+0000","last_active":"2024-04-25T20:57:47.531219+0000","last_peered":"2024-04-25T20:57:47.531219+0000","last_clean":"2024-04-25T20:57:47.531219+0000","last_became_active":"2024-04-25T20:56:51.271224+0000","last_became_peered":"2024-04-25T20:56:51.271224+0000","last_unstale":"2024-04-25T20:57:47.531219+0000","last_undegraded":"2024-04-25T20:57:47.531219+0000","last_fullsized":"2024-04-25T20:57:47.531219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:46.847929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524260+0000","last_change":"2024-04-25T20:56:51.278281+0000","last_active":"2024-04-25T20:57:47.524260+0000","last_peered":"2024-04-25T20:57:47.524260+0000","last_clean":"2024-04-25T20:57:47.524260+0000","last_became_active":"2024-04-25T20:56:51.277947+0000","last_became_peered":"2024-04-25T20:56:51.277947+0000","last_unstale":"2024-04-25T20:57:47.524260+0000","last_undegraded":"2024-04-25T20:57:47.524260+0000","last_fullsized":"2024-04-25T20:57:47.524260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:02.509296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.915415+0000","last_change":"2024-04-25T20:56:51.273784+0000","last_active":"2024-04-25T20:58:45.915415+0000","last_peered":"2024-04-25T20:58:45.915415+0000","last_clean":"2024-04-25T20:58:45.915415+0000","last_became_active":"2024-04-25T20:56:51.273489+0000","last_became_peered":"2024-04-25T20:56:51.273489+0000","last_unstale":"2024-04-25T20:58:45.915415+0000","last_undegraded":"2024-04-25T20:58:45.915415+0000","last_fullsized":"2024-04-25T20:58:45.915415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:00.765186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526297+0000","last_change":"2024-04-25T20:56:51.269829+0000","last_active":"2024-04-25T20:57:47.526297+0000","last_peered":"2024-04-25T20:57:47.526297+0000","last_clean":"2024-04-25T20:57:47.526297+0000","last_became_active":"2024-04-25T20:56:51.269446+0000","last_became_peered":"2024-04-25T20:56:51.269446+0000","last_unstale":"2024-04-25T20:57:47.526297+0000","last_undegraded":"2024-04-25T20:57:47.526297+0000","last_fullsized":"2024-04-25T20:57:47.526297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:12.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526233+0000","last_change":"2024-04-25T20:56:51.278117+0000","last_active":"2024-04-25T20:57:47.526233+0000","last_peered":"2024-04-25T20:57:47.526233+0000","last_clean":"2024-04-25T20:57:47.526233+0000","last_became_active":"2024-04-25T20:56:51.277596+0000","last_became_peered":"2024-04-25T20:56:51.277596+0000","last_unstale":"2024-04-25T20:57:47.526233+0000","last_undegraded":"2024-04-25T20:57:47.526233+0000","last_fullsized":"2024-04-25T20:57:47.526233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:28.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299349+0000","last_change":"2024-04-25T20:56:51.281575+0000","last_active":"2024-04-25T20:57:48.299349+0000","last_peered":"2024-04-25T20:57:48.299349+0000","last_clean":"2024-04-25T20:57:48.299349+0000","last_became_active":"2024-04-25T20:56:51.281217+0000","last_became_peered":"2024-04-25T20:56:51.281217+0000","last_unstale":"2024-04-25T20:57:48.299349+0000","last_undegraded":"2024-04-25T20:57:48.299349+0000","last_fullsized":"2024-04-25T20:57:48.299349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:05.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.528383+0000","last_change":"2024-04-25T20:56:51.284608+0000","last_active":"2024-04-25T20:57:47.528383+0000","last_peered":"2024-04-25T20:57:47.528383+0000","last_clean":"2024-04-25T20:57:47.528383+0000","last_became_active":"2024-04-25T20:56:51.284257+0000","last_became_peered":"2024-04-25T20:56:51.284257+0000","last_unstale":"2024-04-25T20:57:47.528383+0000","last_undegraded":"2024-04-25T20:57:47.528383+0000","last_fullsized":"2024-04-25T20:57:47.528383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:44:59.184192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.523575+0000","last_change":"2024-04-25T20:56:51.276770+0000","last_active":"2024-04-25T20:57:47.523575+0000","last_peered":"2024-04-25T20:57:47.523575+0000","last_clean":"2024-04-25T20:57:47.523575+0000","last_became_active":"2024-04-25T20:56:51.276431+0000","last_became_peered":"2024-04-25T20:56:51.276431+0000","last_unstale":"2024-04-25T20:57:47.523575+0000","last_undegraded":"2024-04-25T20:57:47.523575+0000","last_fullsized":"2024-04-25T20:57:47.523575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:14:25.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524537+0000","last_change":"2024-04-25T20:56:51.283728+0000","last_active":"2024-04-25T20:57:47.524537+0000","last_peered":"2024-04-25T20:57:47.524537+0000","last_clean":"2024-04-25T20:57:47.524537+0000","last_became_active":"2024-04-25T20:56:51.283370+0000","last_became_peered":"2024-04-25T20:56:51.283370+0000","last_unstale":"2024-04-25T20:57:47.524537+0000","last_undegraded":"2024-04-25T20:57:47.524537+0000","last_fullsized":"2024-04-25T20:57:47.524537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:18.042769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.523509+0000","last_change":"2024-04-25T20:56:48.259737+0000","last_active":"2024-04-25T20:57:47.523509+0000","last_peered":"2024-04-25T20:57:47.523509+0000","last_clean":"2024-04-25T20:57:47.523509+0000","last_became_active":"2024-04-25T20:56:48.259442+0000","last_became_peered":"2024-04-25T20:56:48.259442+0000","last_unstale":"2024-04-25T20:57:47.523509+0000","last_undegraded":"2024-04-25T20:57:47.523509+0000","last_fullsized":"2024-04-25T20:57:47.523509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:23.191536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526318+0000","last_change":"2024-04-25T20:56:51.274510+0000","last_active":"2024-04-25T20:57:47.526318+0000","last_peered":"2024-04-25T20:57:47.526318+0000","last_clean":"2024-04-25T20:57:47.526318+0000","last_became_active":"2024-04-25T20:56:51.274137+0000","last_became_peered":"2024-04-25T20:56:51.274137+0000","last_unstale":"2024-04-25T20:57:47.526318+0000","last_undegraded":"2024-04-25T20:57:47.526318+0000","last_fullsized":"2024-04-25T20:57:47.526318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:05.903617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526661+0000","last_change":"2024-04-25T20:56:48.947850+0000","last_active":"2024-04-25T20:57:47.526661+0000","last_peered":"2024-04-25T20:57:47.526661+0000","last_clean":"2024-04-25T20:57:47.526661+0000","last_became_active":"2024-04-25T20:56:48.946160+0000","last_became_peered":"2024-04-25T20:56:48.946160+0000","last_unstale":"2024-04-25T20:57:47.526661+0000","last_undegraded":"2024-04-25T20:57:47.526661+0000","last_fullsized":"2024-04-25T20:57:47.526661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:32.692390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526627+0000","last_change":"2024-04-25T20:56:51.280478+0000","last_active":"2024-04-25T20:57:47.526627+0000","last_peered":"2024-04-25T20:57:47.526627+0000","last_clean":"2024-04-25T20:57:47.526627+0000","last_became_active":"2024-04-25T20:56:51.280173+0000","last_became_peered":"2024-04-25T20:56:51.280173+0000","last_unstale":"2024-04-25T20:57:47.526627+0000","last_undegraded":"2024-04-25T20:57:47.526627+0000","last_fullsized":"2024-04-25T20:57:47.526627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:48.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.535264+0000","last_change":"2024-04-25T20:57:47.535264+0000","last_active":"2024-04-25T20:57:47.535264+0000","last_peered":"2024-04-25T20:57:47.535264+0000","last_clean":"2024-04-25T20:57:47.535264+0000","last_became_active":"2024-04-25T20:57:47.531152+0000","last_became_peered":"2024-04-25T20:57:47.531152+0000","last_unstale":"2024-04-25T20:57:47.535264+0000","last_undegraded":"2024-04-25T20:57:47.535264+0000","last_fullsized":"2024-04-25T20:57:47.535264+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:19.085455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299473+0000","last_change":"2024-04-25T20:56:51.278383+0000","last_active":"2024-04-25T20:57:48.299473+0000","last_peered":"2024-04-25T20:57:48.299473+0000","last_clean":"2024-04-25T20:57:48.299473+0000","last_became_active":"2024-04-25T20:56:51.277603+0000","last_became_peered":"2024-04-25T20:56:51.277603+0000","last_unstale":"2024-04-25T20:57:48.299473+0000","last_undegraded":"2024-04-25T20:57:48.299473+0000","last_fullsized":"2024-04-25T20:57:48.299473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:12.197940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527024+0000","last_change":"2024-04-25T20:56:48.258869+0000","last_active":"2024-04-25T20:57:47.527024+0000","last_peered":"2024-04-25T20:57:47.527024+0000","last_clean":"2024-04-25T20:57:47.527024+0000","last_became_active":"2024-04-25T20:56:48.258414+0000","last_became_peered":"2024-04-25T20:56:48.258414+0000","last_unstale":"2024-04-25T20:57:47.527024+0000","last_undegraded":"2024-04-25T20:57:47.527024+0000","last_fullsized":"2024-04-25T20:57:47.527024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:09.889936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.301047+0000","last_change":"2024-04-25T20:57:48.300637+0000","last_active":"2024-04-25T20:57:48.301047+0000","last_peered":"2024-04-25T20:57:48.301047+0000","last_clean":"2024-04-25T20:57:48.301047+0000","last_became_active":"2024-04-25T20:57:48.300285+0000","last_became_peered":"2024-04-25T20:57:48.300285+0000","last_unstale":"2024-04-25T20:57:48.301047+0000","last_undegraded":"2024-04-25T20:57:48.301047+0000","last_fullsized":"2024-04-25T20:57:48.301047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:31.810671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":129,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:41.876825+0000","last_change":"2024-04-25T20:56:48.262543+0000","last_active":"2024-04-25T20:58:41.876825+0000","last_peered":"2024-04-25T20:58:41.876825+0000","last_clean":"2024-04-25T20:58:41.876825+0000","last_became_active":"2024-04-25T20:56:48.262086+0000","last_became_peered":"2024-04-25T20:56:48.262086+0000","last_unstale":"2024-04-25T20:58:41.876825+0000","last_undegraded":"2024-04-25T20:58:41.876825+0000","last_fullsized":"2024-04-25T20:58:41.876825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:43.722131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526598+0000","last_change":"2024-04-25T20:56:51.273500+0000","last_active":"2024-04-25T20:57:47.526598+0000","last_peered":"2024-04-25T20:57:47.526598+0000","last_clean":"2024-04-25T20:57:47.526598+0000","last_became_active":"2024-04-25T20:56:51.272882+0000","last_became_peered":"2024-04-25T20:56:51.272882+0000","last_unstale":"2024-04-25T20:57:47.526598+0000","last_undegraded":"2024-04-25T20:57:47.526598+0000","last_fullsized":"2024-04-25T20:57:47.526598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:05.682258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793722+0000","last_change":"2024-04-25T20:56:48.255225+0000","last_active":"2024-04-25T20:57:47.793722+0000","last_peered":"2024-04-25T20:57:47.793722+0000","last_clean":"2024-04-25T20:57:47.793722+0000","last_became_active":"2024-04-25T20:56:48.254644+0000","last_became_peered":"2024-04-25T20:56:48.254644+0000","last_unstale":"2024-04-25T20:57:47.793722+0000","last_undegraded":"2024-04-25T20:57:47.793722+0000","last_fullsized":"2024-04-25T20:57:47.793722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:35.724612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591693+0000","last_change":"2024-04-25T20:56:51.270709+0000","last_active":"2024-04-25T20:57:47.591693+0000","last_peered":"2024-04-25T20:57:47.591693+0000","last_clean":"2024-04-25T20:57:47.591693+0000","last_became_active":"2024-04-25T20:56:51.270295+0000","last_became_peered":"2024-04-25T20:56:51.270295+0000","last_unstale":"2024-04-25T20:57:47.591693+0000","last_undegraded":"2024-04-25T20:57:47.591693+0000","last_fullsized":"2024-04-25T20:57:47.591693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:41.998953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793738+0000","last_change":"2024-04-25T20:56:48.256498+0000","last_active":"2024-04-25T20:57:47.793738+0000","last_peered":"2024-04-25T20:57:47.793738+0000","last_clean":"2024-04-25T20:57:47.793738+0000","last_became_active":"2024-04-25T20:56:48.256140+0000","last_became_peered":"2024-04-25T20:56:48.256140+0000","last_unstale":"2024-04-25T20:57:47.793738+0000","last_undegraded":"2024-04-25T20:57:47.793738+0000","last_fullsized":"2024-04-25T20:57:47.793738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:41.458235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.525453+0000","last_change":"2024-04-25T20:56:51.267575+0000","last_active":"2024-04-25T20:57:47.525453+0000","last_peered":"2024-04-25T20:57:47.525453+0000","last_clean":"2024-04-25T20:57:47.525453+0000","last_became_active":"2024-04-25T20:56:51.267265+0000","last_became_peered":"2024-04-25T20:56:51.267265+0000","last_unstale":"2024-04-25T20:57:47.525453+0000","last_undegraded":"2024-04-25T20:57:47.525453+0000","last_fullsized":"2024-04-25T20:57:47.525453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:30.760555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591411+0000","last_change":"2024-04-25T20:56:48.260747+0000","last_active":"2024-04-25T20:57:47.591411+0000","last_peered":"2024-04-25T20:57:47.591411+0000","last_clean":"2024-04-25T20:57:47.591411+0000","last_became_active":"2024-04-25T20:56:48.260339+0000","last_became_peered":"2024-04-25T20:56:48.260339+0000","last_unstale":"2024-04-25T20:57:47.591411+0000","last_undegraded":"2024-04-25T20:57:47.591411+0000","last_fullsized":"2024-04-25T20:57:47.591411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:19.769631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.795125+0000","last_change":"2024-04-25T20:57:47.795125+0000","last_active":"2024-04-25T20:57:47.795125+0000","last_peered":"2024-04-25T20:57:47.795125+0000","last_clean":"2024-04-25T20:57:47.795125+0000","last_became_active":"2024-04-25T20:57:47.794642+0000","last_became_peered":"2024-04-25T20:57:47.794642+0000","last_unstale":"2024-04-25T20:57:47.795125+0000","last_undegraded":"2024-04-25T20:57:47.795125+0000","last_fullsized":"2024-04-25T20:57:47.795125+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:15.443731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:43.879294+0000","last_change":"2024-04-25T20:56:48.256113+0000","last_active":"2024-04-25T20:58:43.879294+0000","last_peered":"2024-04-25T20:58:43.879294+0000","last_clean":"2024-04-25T20:58:43.879294+0000","last_became_active":"2024-04-25T20:56:48.255644+0000","last_became_peered":"2024-04-25T20:56:48.255644+0000","last_unstale":"2024-04-25T20:58:43.879294+0000","last_undegraded":"2024-04-25T20:58:43.879294+0000","last_fullsized":"2024-04-25T20:58:43.879294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:12.100099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299841+0000","last_change":"2024-04-25T20:56:51.276990+0000","last_active":"2024-04-25T20:57:48.299841+0000","last_peered":"2024-04-25T20:57:48.299841+0000","last_clean":"2024-04-25T20:57:48.299841+0000","last_became_active":"2024-04-25T20:56:51.276638+0000","last_became_peered":"2024-04-25T20:56:51.276638+0000","last_unstale":"2024-04-25T20:57:48.299841+0000","last_undegraded":"2024-04-25T20:57:48.299841+0000","last_fullsized":"2024-04-25T20:57:48.299841+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:06.645278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531382+0000","last_change":"2024-04-25T20:56:48.263216+0000","last_active":"2024-04-25T20:57:47.531382+0000","last_peered":"2024-04-25T20:57:47.531382+0000","last_clean":"2024-04-25T20:57:47.531382+0000","last_became_active":"2024-04-25T20:56:48.262854+0000","last_became_peered":"2024-04-25T20:56:48.262854+0000","last_unstale":"2024-04-25T20:57:47.531382+0000","last_undegraded":"2024-04-25T20:57:47.531382+0000","last_fullsized":"2024-04-25T20:57:47.531382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:00.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526518+0000","last_change":"2024-04-25T20:56:51.278479+0000","last_active":"2024-04-25T20:57:47.526518+0000","last_peered":"2024-04-25T20:57:47.526518+0000","last_clean":"2024-04-25T20:57:47.526518+0000","last_became_active":"2024-04-25T20:56:51.276285+0000","last_became_peered":"2024-04-25T20:56:51.276285+0000","last_unstale":"2024-04-25T20:57:47.526518+0000","last_undegraded":"2024-04-25T20:57:47.526518+0000","last_fullsized":"2024-04-25T20:57:47.526518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:14.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591683+0000","last_change":"2024-04-25T20:56:48.946808+0000","last_active":"2024-04-25T20:57:47.591683+0000","last_peered":"2024-04-25T20:57:47.591683+0000","last_clean":"2024-04-25T20:57:47.591683+0000","last_became_active":"2024-04-25T20:56:48.946421+0000","last_became_peered":"2024-04-25T20:56:48.946421+0000","last_unstale":"2024-04-25T20:57:47.591683+0000","last_undegraded":"2024-04-25T20:57:47.591683+0000","last_fullsized":"2024-04-25T20:57:47.591683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:47.763950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299666+0000","last_change":"2024-04-25T20:56:51.274761+0000","last_active":"2024-04-25T20:57:48.299666+0000","last_peered":"2024-04-25T20:57:48.299666+0000","last_clean":"2024-04-25T20:57:48.299666+0000","last_became_active":"2024-04-25T20:56:51.274452+0000","last_became_peered":"2024-04-25T20:56:51.274452+0000","last_unstale":"2024-04-25T20:57:48.299666+0000","last_undegraded":"2024-04-25T20:57:48.299666+0000","last_fullsized":"2024-04-25T20:57:48.299666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:32.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'28","reported_seq":257,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:44.880941+0000","last_change":"2024-04-25T20:56:48.258982+0000","last_active":"2024-04-25T20:58:44.880941+0000","last_peered":"2024-04-25T20:58:44.880941+0000","last_clean":"2024-04-25T20:58:44.880941+0000","last_became_active":"2024-04-25T20:56:48.258593+0000","last_became_peered":"2024-04-25T20:56:48.258593+0000","last_unstale":"2024-04-25T20:58:44.880941+0000","last_undegraded":"2024-04-25T20:58:44.880941+0000","last_fullsized":"2024-04-25T20:58:44.880941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:20.518202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":203,"num_read_kb":160,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527411+0000","last_change":"2024-04-25T20:56:51.276361+0000","last_active":"2024-04-25T20:57:47.527411+0000","last_peered":"2024-04-25T20:57:47.527411+0000","last_clean":"2024-04-25T20:57:47.527411+0000","last_became_active":"2024-04-25T20:56:51.276045+0000","last_became_peered":"2024-04-25T20:56:51.276045+0000","last_unstale":"2024-04-25T20:57:47.527411+0000","last_undegraded":"2024-04-25T20:57:47.527411+0000","last_fullsized":"2024-04-25T20:57:47.527411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:04.325068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524318+0000","last_change":"2024-04-25T20:56:48.252358+0000","last_active":"2024-04-25T20:57:47.524318+0000","last_peered":"2024-04-25T20:57:47.524318+0000","last_clean":"2024-04-25T20:57:47.524318+0000","last_became_active":"2024-04-25T20:56:48.252027+0000","last_became_peered":"2024-04-25T20:56:48.252027+0000","last_unstale":"2024-04-25T20:57:47.524318+0000","last_undegraded":"2024-04-25T20:57:47.524318+0000","last_fullsized":"2024-04-25T20:57:47.524318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:03.024487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.910850+0000","last_change":"2024-04-25T20:56:51.279599+0000","last_active":"2024-04-25T20:58:45.910850+0000","last_peered":"2024-04-25T20:58:45.910850+0000","last_clean":"2024-04-25T20:58:45.910850+0000","last_became_active":"2024-04-25T20:56:51.279294+0000","last_became_peered":"2024-04-25T20:56:51.279294+0000","last_unstale":"2024-04-25T20:58:45.910850+0000","last_undegraded":"2024-04-25T20:58:45.910850+0000","last_fullsized":"2024-04-25T20:58:45.910850+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591497+0000","last_change":"2024-04-25T20:56:48.261005+0000","last_active":"2024-04-25T20:57:47.591497+0000","last_peered":"2024-04-25T20:57:47.591497+0000","last_clean":"2024-04-25T20:57:47.591497+0000","last_became_active":"2024-04-25T20:56:48.260543+0000","last_became_peered":"2024-04-25T20:56:48.260543+0000","last_unstale":"2024-04-25T20:57:47.591497+0000","last_undegraded":"2024-04-25T20:57:47.591497+0000","last_fullsized":"2024-04-25T20:57:47.591497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:41:54.326036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531044+0000","last_change":"2024-04-25T20:56:51.271753+0000","last_active":"2024-04-25T20:57:47.531044+0000","last_peered":"2024-04-25T20:57:47.531044+0000","last_clean":"2024-04-25T20:57:47.531044+0000","last_became_active":"2024-04-25T20:56:51.271392+0000","last_became_peered":"2024-04-25T20:56:51.271392+0000","last_unstale":"2024-04-25T20:57:47.531044+0000","last_undegraded":"2024-04-25T20:57:47.531044+0000","last_fullsized":"2024-04-25T20:57:47.531044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:26.605647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":127,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:41.876654+0000","last_change":"2024-04-25T20:56:48.260729+0000","last_active":"2024-04-25T20:58:41.876654+0000","last_peered":"2024-04-25T20:58:41.876654+0000","last_clean":"2024-04-25T20:58:41.876654+0000","last_became_active":"2024-04-25T20:56:48.260377+0000","last_became_peered":"2024-04-25T20:56:48.260377+0000","last_unstale":"2024-04-25T20:58:41.876654+0000","last_undegraded":"2024-04-25T20:58:41.876654+0000","last_fullsized":"2024-04-25T20:58:41.876654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:21.582266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.530714+0000","last_change":"2024-04-25T20:56:51.275691+0000","last_active":"2024-04-25T20:57:47.530714+0000","last_peered":"2024-04-25T20:57:47.530714+0000","last_clean":"2024-04-25T20:57:47.530714+0000","last_became_active":"2024-04-25T20:56:51.275220+0000","last_became_peered":"2024-04-25T20:56:51.275220+0000","last_unstale":"2024-04-25T20:57:47.530714+0000","last_undegraded":"2024-04-25T20:57:47.530714+0000","last_fullsized":"2024-04-25T20:57:47.530714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:24.367413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526402+0000","last_change":"2024-04-25T20:56:48.946465+0000","last_active":"2024-04-25T20:57:47.526402+0000","last_peered":"2024-04-25T20:57:47.526402+0000","last_clean":"2024-04-25T20:57:47.526402+0000","last_became_active":"2024-04-25T20:56:48.946093+0000","last_became_peered":"2024-04-25T20:56:48.946093+0000","last_unstale":"2024-04-25T20:57:47.526402+0000","last_undegraded":"2024-04-25T20:57:47.526402+0000","last_fullsized":"2024-04-25T20:57:47.526402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:29.502073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591426+0000","last_change":"2024-04-25T20:56:51.278464+0000","last_active":"2024-04-25T20:57:47.591426+0000","last_peered":"2024-04-25T20:57:47.591426+0000","last_clean":"2024-04-25T20:57:47.591426+0000","last_became_active":"2024-04-25T20:56:51.278119+0000","last_became_peered":"2024-04-25T20:56:51.278119+0000","last_unstale":"2024-04-25T20:57:47.591426+0000","last_undegraded":"2024-04-25T20:57:47.591426+0000","last_fullsized":"2024-04-25T20:57:47.591426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:57:37.224817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.528952+0000","last_change":"2024-04-25T20:56:48.255123+0000","last_active":"2024-04-25T20:57:47.528952+0000","last_peered":"2024-04-25T20:57:47.528952+0000","last_clean":"2024-04-25T20:57:47.528952+0000","last_became_active":"2024-04-25T20:56:48.254680+0000","last_became_peered":"2024-04-25T20:56:48.254680+0000","last_unstale":"2024-04-25T20:57:47.528952+0000","last_undegraded":"2024-04-25T20:57:47.528952+0000","last_fullsized":"2024-04-25T20:57:47.528952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:24.089590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.592950+0000","last_change":"2024-04-25T20:57:47.592950+0000","last_active":"2024-04-25T20:57:47.592950+0000","last_peered":"2024-04-25T20:57:47.592950+0000","last_clean":"2024-04-25T20:57:47.592950+0000","last_became_active":"2024-04-25T20:57:47.592486+0000","last_became_peered":"2024-04-25T20:57:47.592486+0000","last_unstale":"2024-04-25T20:57:47.592950+0000","last_undegraded":"2024-04-25T20:57:47.592950+0000","last_fullsized":"2024-04-25T20:57:47.592950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:43.218061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.523705+0000","last_change":"2024-04-25T20:56:48.947508+0000","last_active":"2024-04-25T20:57:47.523705+0000","last_peered":"2024-04-25T20:57:47.523705+0000","last_clean":"2024-04-25T20:57:47.523705+0000","last_became_active":"2024-04-25T20:56:48.946582+0000","last_became_peered":"2024-04-25T20:56:48.946582+0000","last_unstale":"2024-04-25T20:57:47.523705+0000","last_undegraded":"2024-04-25T20:57:47.523705+0000","last_fullsized":"2024-04-25T20:57:47.523705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:11.198667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793925+0000","last_change":"2024-04-25T20:56:51.273560+0000","last_active":"2024-04-25T20:57:47.793925+0000","last_peered":"2024-04-25T20:57:47.793925+0000","last_clean":"2024-04-25T20:57:47.793925+0000","last_became_active":"2024-04-25T20:56:51.273182+0000","last_became_peered":"2024-04-25T20:56:51.273182+0000","last_unstale":"2024-04-25T20:57:47.793925+0000","last_undegraded":"2024-04-25T20:57:47.793925+0000","last_fullsized":"2024-04-25T20:57:47.793925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:47.012771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531232+0000","last_change":"2024-04-25T20:56:48.260754+0000","last_active":"2024-04-25T20:57:47.531232+0000","last_peered":"2024-04-25T20:57:47.531232+0000","last_clean":"2024-04-25T20:57:47.531232+0000","last_became_active":"2024-04-25T20:56:48.260420+0000","last_became_peered":"2024-04-25T20:56:48.260420+0000","last_unstale":"2024-04-25T20:57:47.531232+0000","last_undegraded":"2024-04-25T20:57:47.531232+0000","last_fullsized":"2024-04-25T20:57:47.531232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:00.219023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793815+0000","last_change":"2024-04-25T20:56:51.271944+0000","last_active":"2024-04-25T20:57:47.793815+0000","last_peered":"2024-04-25T20:57:47.793815+0000","last_clean":"2024-04-25T20:57:47.793815+0000","last_became_active":"2024-04-25T20:56:51.270656+0000","last_became_peered":"2024-04-25T20:56:51.270656+0000","last_unstale":"2024-04-25T20:57:47.793815+0000","last_undegraded":"2024-04-25T20:57:47.793815+0000","last_fullsized":"2024-04-25T20:57:47.793815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:00.624888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527056+0000","last_change":"2024-04-25T20:57:47.526172+0000","last_active":"2024-04-25T20:57:47.527056+0000","last_peered":"2024-04-25T20:57:47.527056+0000","last_clean":"2024-04-25T20:57:47.527056+0000","last_became_active":"2024-04-25T20:57:47.525820+0000","last_became_peered":"2024-04-25T20:57:47.525820+0000","last_unstale":"2024-04-25T20:57:47.527056+0000","last_undegraded":"2024-04-25T20:57:47.527056+0000","last_fullsized":"2024-04-25T20:57:47.527056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:56.458959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":129,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.878953+0000","last_change":"2024-04-25T20:56:48.946471+0000","last_active":"2024-04-25T20:58:45.878953+0000","last_peered":"2024-04-25T20:58:45.878953+0000","last_clean":"2024-04-25T20:58:45.878953+0000","last_became_active":"2024-04-25T20:56:48.946029+0000","last_became_peered":"2024-04-25T20:56:48.946029+0000","last_unstale":"2024-04-25T20:58:45.878953+0000","last_undegraded":"2024-04-25T20:58:45.878953+0000","last_fullsized":"2024-04-25T20:58:45.878953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:30:16.400157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.301165+0000","last_change":"2024-04-25T20:57:48.300758+0000","last_active":"2024-04-25T20:57:48.301165+0000","last_peered":"2024-04-25T20:57:48.301165+0000","last_clean":"2024-04-25T20:57:48.301165+0000","last_became_active":"2024-04-25T20:57:48.300328+0000","last_became_peered":"2024-04-25T20:57:48.300328+0000","last_unstale":"2024-04-25T20:57:48.301165+0000","last_undegraded":"2024-04-25T20:57:48.301165+0000","last_fullsized":"2024-04-25T20:57:48.301165+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:31.931834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524170+0000","last_change":"2024-04-25T20:56:48.258479+0000","last_active":"2024-04-25T20:57:47.524170+0000","last_peered":"2024-04-25T20:57:47.524170+0000","last_clean":"2024-04-25T20:57:47.524170+0000","last_became_active":"2024-04-25T20:56:48.257951+0000","last_became_peered":"2024-04-25T20:56:48.257951+0000","last_unstale":"2024-04-25T20:57:47.524170+0000","last_undegraded":"2024-04-25T20:57:47.524170+0000","last_fullsized":"2024-04-25T20:57:47.524170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:13.812201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.530900+0000","last_change":"2024-04-25T20:55:28.701357+0000","last_active":"2024-04-25T20:57:47.530900+0000","last_peered":"2024-04-25T20:57:47.530900+0000","last_clean":"2024-04-25T20:57:47.530900+0000","last_became_active":"2024-04-25T20:55:28.700906+0000","last_became_peered":"2024-04-25T20:55:28.700906+0000","last_unstale":"2024-04-25T20:57:47.530900+0000","last_undegraded":"2024-04-25T20:57:47.530900+0000","last_fullsized":"2024-04-25T20:57:47.530900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_clean_scrub_stamp":"2024-04-25T20:52:10.334323+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:37:35.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.907807+0000","last_change":"2024-04-25T20:56:51.275249+0000","last_active":"2024-04-25T20:58:45.907807+0000","last_peered":"2024-04-25T20:58:45.907807+0000","last_clean":"2024-04-25T20:58:45.907807+0000","last_became_active":"2024-04-25T20:56:51.274938+0000","last_became_peered":"2024-04-25T20:56:51.274938+0000","last_unstale":"2024-04-25T20:58:45.907807+0000","last_undegraded":"2024-04-25T20:58:45.907807+0000","last_fullsized":"2024-04-25T20:58:45.907807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:53.933324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531037+0000","last_change":"2024-04-25T20:56:48.259357+0000","last_active":"2024-04-25T20:57:47.531037+0000","last_peered":"2024-04-25T20:57:47.531037+0000","last_clean":"2024-04-25T20:57:47.531037+0000","last_became_active":"2024-04-25T20:56:48.258840+0000","last_became_peered":"2024-04-25T20:56:48.258840+0000","last_unstale":"2024-04-25T20:57:47.531037+0000","last_undegraded":"2024-04-25T20:57:47.531037+0000","last_fullsized":"2024-04-25T20:57:47.531037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:30.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527510+0000","last_change":"2024-04-25T20:56:48.946498+0000","last_active":"2024-04-25T20:57:47.527510+0000","last_peered":"2024-04-25T20:57:47.527510+0000","last_clean":"2024-04-25T20:57:47.527510+0000","last_became_active":"2024-04-25T20:56:48.946124+0000","last_became_peered":"2024-04-25T20:56:48.946124+0000","last_unstale":"2024-04-25T20:57:47.527510+0000","last_undegraded":"2024-04-25T20:57:47.527510+0000","last_fullsized":"2024-04-25T20:57:47.527510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:08.545574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.908819+0000","last_change":"2024-04-25T20:56:51.276093+0000","last_active":"2024-04-25T20:58:45.908819+0000","last_peered":"2024-04-25T20:58:45.908819+0000","last_clean":"2024-04-25T20:58:45.908819+0000","last_became_active":"2024-04-25T20:56:51.275771+0000","last_became_peered":"2024-04-25T20:56:51.275771+0000","last_unstale":"2024-04-25T20:58:45.908819+0000","last_undegraded":"2024-04-25T20:58:45.908819+0000","last_fullsized":"2024-04-25T20:58:45.908819+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:15.159208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.533338+0000","last_change":"2024-04-25T20:57:47.533338+0000","last_active":"2024-04-25T20:57:47.533338+0000","last_peered":"2024-04-25T20:57:47.533338+0000","last_clean":"2024-04-25T20:57:47.533338+0000","last_became_active":"2024-04-25T20:57:47.529539+0000","last_became_peered":"2024-04-25T20:57:47.529539+0000","last_unstale":"2024-04-25T20:57:47.533338+0000","last_undegraded":"2024-04-25T20:57:47.533338+0000","last_fullsized":"2024-04-25T20:57:47.533338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:25.099128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526208+0000","last_change":"2024-04-25T20:56:51.275461+0000","last_active":"2024-04-25T20:57:47.526208+0000","last_peered":"2024-04-25T20:57:47.526208+0000","last_clean":"2024-04-25T20:57:47.526208+0000","last_became_active":"2024-04-25T20:56:51.275087+0000","last_became_peered":"2024-04-25T20:56:51.275087+0000","last_unstale":"2024-04-25T20:57:47.526208+0000","last_undegraded":"2024-04-25T20:57:47.526208+0000","last_fullsized":"2024-04-25T20:57:47.526208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:01:02.514490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524270+0000","last_change":"2024-04-25T20:56:49.031177+0000","last_active":"2024-04-25T20:57:47.524270+0000","last_peered":"2024-04-25T20:57:47.524270+0000","last_clean":"2024-04-25T20:57:47.524270+0000","last_became_active":"2024-04-25T20:56:49.030697+0000","last_became_peered":"2024-04-25T20:56:49.030697+0000","last_unstale":"2024-04-25T20:57:47.524270+0000","last_undegraded":"2024-04-25T20:57:47.524270+0000","last_fullsized":"2024-04-25T20:57:47.524270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:13.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.795434+0000","last_change":"2024-04-25T20:57:47.795434+0000","last_active":"2024-04-25T20:57:47.795434+0000","last_peered":"2024-04-25T20:57:47.795434+0000","last_clean":"2024-04-25T20:57:47.795434+0000","last_became_active":"2024-04-25T20:57:47.795036+0000","last_became_peered":"2024-04-25T20:57:47.795036+0000","last_unstale":"2024-04-25T20:57:47.795434+0000","last_undegraded":"2024-04-25T20:57:47.795434+0000","last_fullsized":"2024-04-25T20:57:47.795434+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:48:56.749328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299385+0000","last_change":"2024-04-25T20:56:48.946689+0000","last_active":"2024-04-25T20:57:48.299385+0000","last_peered":"2024-04-25T20:57:48.299385+0000","last_clean":"2024-04-25T20:57:48.299385+0000","last_became_active":"2024-04-25T20:56:48.946229+0000","last_became_peered":"2024-04-25T20:56:48.946229+0000","last_unstale":"2024-04-25T20:57:48.299385+0000","last_undegraded":"2024-04-25T20:57:48.299385+0000","last_fullsized":"2024-04-25T20:57:48.299385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:04.120197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526955+0000","last_change":"2024-04-25T20:56:51.275657+0000","last_active":"2024-04-25T20:57:47.526955+0000","last_peered":"2024-04-25T20:57:47.526955+0000","last_clean":"2024-04-25T20:57:47.526955+0000","last_became_active":"2024-04-25T20:56:51.275237+0000","last_became_peered":"2024-04-25T20:56:51.275237+0000","last_unstale":"2024-04-25T20:57:47.526955+0000","last_undegraded":"2024-04-25T20:57:47.526955+0000","last_fullsized":"2024-04-25T20:57:47.526955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:39.778340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.525606+0000","last_change":"2024-04-25T20:56:48.251882+0000","last_active":"2024-04-25T20:57:47.525606+0000","last_peered":"2024-04-25T20:57:47.525606+0000","last_clean":"2024-04-25T20:57:47.525606+0000","last_became_active":"2024-04-25T20:56:48.251571+0000","last_became_peered":"2024-04-25T20:56:48.251571+0000","last_unstale":"2024-04-25T20:57:47.525606+0000","last_undegraded":"2024-04-25T20:57:47.525606+0000","last_fullsized":"2024-04-25T20:57:47.525606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:51.769332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527481+0000","last_change":"2024-04-25T20:56:51.280648+0000","last_active":"2024-04-25T20:57:47.527481+0000","last_peered":"2024-04-25T20:57:47.527481+0000","last_clean":"2024-04-25T20:57:47.527481+0000","last_became_active":"2024-04-25T20:56:51.280232+0000","last_became_peered":"2024-04-25T20:56:51.280232+0000","last_unstale":"2024-04-25T20:57:47.527481+0000","last_undegraded":"2024-04-25T20:57:47.527481+0000","last_fullsized":"2024-04-25T20:57:47.527481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:12.375519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.535653+0000","last_change":"2024-04-25T20:57:47.535216+0000","last_active":"2024-04-25T20:57:47.535653+0000","last_peered":"2024-04-25T20:57:47.535653+0000","last_clean":"2024-04-25T20:57:47.535653+0000","last_became_active":"2024-04-25T20:57:47.530348+0000","last_became_peered":"2024-04-25T20:57:47.530348+0000","last_unstale":"2024-04-25T20:57:47.535653+0000","last_undegraded":"2024-04-25T20:57:47.535653+0000","last_fullsized":"2024-04-25T20:57:47.535653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:01.373541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.525758+0000","last_change":"2024-04-25T20:56:51.277166+0000","last_active":"2024-04-25T20:57:47.525758+0000","last_peered":"2024-04-25T20:57:47.525758+0000","last_clean":"2024-04-25T20:57:47.525758+0000","last_became_active":"2024-04-25T20:56:51.276840+0000","last_became_peered":"2024-04-25T20:56:51.276840+0000","last_unstale":"2024-04-25T20:57:47.525758+0000","last_undegraded":"2024-04-25T20:57:47.525758+0000","last_fullsized":"2024-04-25T20:57:47.525758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:52.756660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526676+0000","last_change":"2024-04-25T20:56:48.263426+0000","last_active":"2024-04-25T20:57:47.526676+0000","last_peered":"2024-04-25T20:57:47.526676+0000","last_clean":"2024-04-25T20:57:47.526676+0000","last_became_active":"2024-04-25T20:56:48.262832+0000","last_became_peered":"2024-04-25T20:56:48.262832+0000","last_unstale":"2024-04-25T20:57:47.526676+0000","last_undegraded":"2024-04-25T20:57:47.526676+0000","last_fullsized":"2024-04-25T20:57:47.526676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:09.362161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531322+0000","last_change":"2024-04-25T20:56:51.274267+0000","last_active":"2024-04-25T20:57:47.531322+0000","last_peered":"2024-04-25T20:57:47.531322+0000","last_clean":"2024-04-25T20:57:47.531322+0000","last_became_active":"2024-04-25T20:56:51.273807+0000","last_became_peered":"2024-04-25T20:56:51.273807+0000","last_unstale":"2024-04-25T20:57:47.531322+0000","last_undegraded":"2024-04-25T20:57:47.531322+0000","last_fullsized":"2024-04-25T20:57:47.531322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524443+0000","last_change":"2024-04-25T20:56:49.031011+0000","last_active":"2024-04-25T20:57:47.524443+0000","last_peered":"2024-04-25T20:57:47.524443+0000","last_clean":"2024-04-25T20:57:47.524443+0000","last_became_active":"2024-04-25T20:56:49.030522+0000","last_became_peered":"2024-04-25T20:56:49.030522+0000","last_unstale":"2024-04-25T20:57:47.524443+0000","last_undegraded":"2024-04-25T20:57:47.524443+0000","last_fullsized":"2024-04-25T20:57:47.524443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:00.915454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531323+0000","last_change":"2024-04-25T20:56:51.276989+0000","last_active":"2024-04-25T20:57:47.531323+0000","last_peered":"2024-04-25T20:57:47.531323+0000","last_clean":"2024-04-25T20:57:47.531323+0000","last_became_active":"2024-04-25T20:56:51.275417+0000","last_became_peered":"2024-04-25T20:56:51.275417+0000","last_unstale":"2024-04-25T20:57:47.531323+0000","last_undegraded":"2024-04-25T20:57:47.531323+0000","last_fullsized":"2024-04-25T20:57:47.531323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:27.516062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591582+0000","last_change":"2024-04-25T20:56:48.253713+0000","last_active":"2024-04-25T20:57:47.591582+0000","last_peered":"2024-04-25T20:57:47.591582+0000","last_clean":"2024-04-25T20:57:47.591582+0000","last_became_active":"2024-04-25T20:56:48.253215+0000","last_became_peered":"2024-04-25T20:56:48.253215+0000","last_unstale":"2024-04-25T20:57:47.591582+0000","last_undegraded":"2024-04-25T20:57:47.591582+0000","last_fullsized":"2024-04-25T20:57:47.591582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:57.946840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531025+0000","last_change":"2024-04-25T20:56:51.274691+0000","last_active":"2024-04-25T20:57:47.531025+0000","last_peered":"2024-04-25T20:57:47.531025+0000","last_clean":"2024-04-25T20:57:47.531025+0000","last_became_active":"2024-04-25T20:56:51.274350+0000","last_became_peered":"2024-04-25T20:56:51.274350+0000","last_unstale":"2024-04-25T20:57:47.531025+0000","last_undegraded":"2024-04-25T20:57:47.531025+0000","last_fullsized":"2024-04-25T20:57:47.531025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:52.375409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.794071+0000","last_change":"2024-04-25T20:56:48.251616+0000","last_active":"2024-04-25T20:57:47.794071+0000","last_peered":"2024-04-25T20:57:47.794071+0000","last_clean":"2024-04-25T20:57:47.794071+0000","last_became_active":"2024-04-25T20:56:48.251248+0000","last_became_peered":"2024-04-25T20:56:48.251248+0000","last_unstale":"2024-04-25T20:57:47.794071+0000","last_undegraded":"2024-04-25T20:57:47.794071+0000","last_fullsized":"2024-04-25T20:57:47.794071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:51.032356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531276+0000","last_change":"2024-04-25T20:56:51.278372+0000","last_active":"2024-04-25T20:57:47.531276+0000","last_peered":"2024-04-25T20:57:47.531276+0000","last_clean":"2024-04-25T20:57:47.531276+0000","last_became_active":"2024-04-25T20:56:51.277989+0000","last_became_peered":"2024-04-25T20:56:51.277989+0000","last_unstale":"2024-04-25T20:57:47.531276+0000","last_undegraded":"2024-04-25T20:57:47.531276+0000","last_fullsized":"2024-04-25T20:57:47.531276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.017713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527533+0000","last_change":"2024-04-25T20:56:51.276270+0000","last_active":"2024-04-25T20:57:47.527533+0000","last_peered":"2024-04-25T20:57:47.527533+0000","last_clean":"2024-04-25T20:57:47.527533+0000","last_became_active":"2024-04-25T20:56:51.275948+0000","last_became_peered":"2024-04-25T20:56:51.275948+0000","last_unstale":"2024-04-25T20:57:47.527533+0000","last_undegraded":"2024-04-25T20:57:47.527533+0000","last_fullsized":"2024-04-25T20:57:47.527533+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:33.548439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524213+0000","last_change":"2024-04-25T20:56:51.283931+0000","last_active":"2024-04-25T20:57:47.524213+0000","last_peered":"2024-04-25T20:57:47.524213+0000","last_clean":"2024-04-25T20:57:47.524213+0000","last_became_active":"2024-04-25T20:56:51.283120+0000","last_became_peered":"2024-04-25T20:56:51.283120+0000","last_unstale":"2024-04-25T20:57:47.524213+0000","last_undegraded":"2024-04-25T20:57:47.524213+0000","last_fullsized":"2024-04-25T20:57:47.524213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:26.676750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524219+0000","last_change":"2024-04-25T20:56:51.286013+0000","last_active":"2024-04-25T20:57:47.524219+0000","last_peered":"2024-04-25T20:57:47.524219+0000","last_clean":"2024-04-25T20:57:47.524219+0000","last_became_active":"2024-04-25T20:56:51.285645+0000","last_became_peered":"2024-04-25T20:56:51.285645+0000","last_unstale":"2024-04-25T20:57:47.524219+0000","last_undegraded":"2024-04-25T20:57:47.524219+0000","last_fullsized":"2024-04-25T20:57:47.524219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:51.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526615+0000","last_change":"2024-04-25T20:56:51.272567+0000","last_active":"2024-04-25T20:57:47.526615+0000","last_peered":"2024-04-25T20:57:47.526615+0000","last_clean":"2024-04-25T20:57:47.526615+0000","last_became_active":"2024-04-25T20:56:51.271427+0000","last_became_peered":"2024-04-25T20:56:51.271427+0000","last_unstale":"2024-04-25T20:57:47.526615+0000","last_undegraded":"2024-04-25T20:57:47.526615+0000","last_fullsized":"2024-04-25T20:57:47.526615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:19.945982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793917+0000","last_change":"2024-04-25T20:56:51.272448+0000","last_active":"2024-04-25T20:57:47.793917+0000","last_peered":"2024-04-25T20:57:47.793917+0000","last_clean":"2024-04-25T20:57:47.793917+0000","last_became_active":"2024-04-25T20:56:51.271965+0000","last_became_peered":"2024-04-25T20:56:51.271965+0000","last_unstale":"2024-04-25T20:57:47.793917+0000","last_undegraded":"2024-04-25T20:57:47.793917+0000","last_fullsized":"2024-04-25T20:57:47.793917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:03.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527715+0000","last_change":"2024-04-25T20:56:51.280260+0000","last_active":"2024-04-25T20:57:47.527715+0000","last_peered":"2024-04-25T20:57:47.527715+0000","last_clean":"2024-04-25T20:57:47.527715+0000","last_became_active":"2024-04-25T20:56:51.279864+0000","last_became_peered":"2024-04-25T20:56:51.279864+0000","last_unstale":"2024-04-25T20:57:47.527715+0000","last_undegraded":"2024-04-25T20:57:47.527715+0000","last_fullsized":"2024-04-25T20:57:47.527715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.136824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524461+0000","last_change":"2024-04-25T20:56:51.279041+0000","last_active":"2024-04-25T20:57:47.524461+0000","last_peered":"2024-04-25T20:57:47.524461+0000","last_clean":"2024-04-25T20:57:47.524461+0000","last_became_active":"2024-04-25T20:56:51.278302+0000","last_became_peered":"2024-04-25T20:56:51.278302+0000","last_unstale":"2024-04-25T20:57:47.524461+0000","last_undegraded":"2024-04-25T20:57:47.524461+0000","last_fullsized":"2024-04-25T20:57:47.524461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:02.510919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526093+0000","last_change":"2024-04-25T20:56:51.275618+0000","last_active":"2024-04-25T20:57:47.526093+0000","last_peered":"2024-04-25T20:57:47.526093+0000","last_clean":"2024-04-25T20:57:47.526093+0000","last_became_active":"2024-04-25T20:56:51.275249+0000","last_became_peered":"2024-04-25T20:56:51.275249+0000","last_unstale":"2024-04-25T20:57:47.526093+0000","last_undegraded":"2024-04-25T20:57:47.526093+0000","last_fullsized":"2024-04-25T20:57:47.526093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:01.793083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591548+0000","last_change":"2024-04-25T20:56:51.275868+0000","last_active":"2024-04-25T20:57:47.591548+0000","last_peered":"2024-04-25T20:57:47.591548+0000","last_clean":"2024-04-25T20:57:47.591548+0000","last_became_active":"2024-04-25T20:56:51.275544+0000","last_became_peered":"2024-04-25T20:56:51.275544+0000","last_unstale":"2024-04-25T20:57:47.591548+0000","last_undegraded":"2024-04-25T20:57:47.591548+0000","last_fullsized":"2024-04-25T20:57:47.591548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:31.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.525459+0000","last_change":"2024-04-25T20:56:51.274963+0000","last_active":"2024-04-25T20:57:47.525459+0000","last_peered":"2024-04-25T20:57:47.525459+0000","last_clean":"2024-04-25T20:57:47.525459+0000","last_became_active":"2024-04-25T20:56:51.274597+0000","last_became_peered":"2024-04-25T20:56:51.274597+0000","last_unstale":"2024-04-25T20:57:47.525459+0000","last_undegraded":"2024-04-25T20:57:47.525459+0000","last_fullsized":"2024-04-25T20:57:47.525459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:14.464560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299653+0000","last_change":"2024-04-25T20:56:51.279602+0000","last_active":"2024-04-25T20:57:48.299653+0000","last_peered":"2024-04-25T20:57:48.299653+0000","last_clean":"2024-04-25T20:57:48.299653+0000","last_became_active":"2024-04-25T20:56:51.279221+0000","last_became_peered":"2024-04-25T20:56:51.279221+0000","last_unstale":"2024-04-25T20:57:48.299653+0000","last_undegraded":"2024-04-25T20:57:48.299653+0000","last_fullsized":"2024-04-25T20:57:48.299653+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:26.960492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.910363+0000","last_change":"2024-04-25T20:56:51.280465+0000","last_active":"2024-04-25T20:58:45.910363+0000","last_peered":"2024-04-25T20:58:45.910363+0000","last_clean":"2024-04-25T20:58:45.910363+0000","last_became_active":"2024-04-25T20:56:51.280168+0000","last_became_peered":"2024-04-25T20:56:51.280168+0000","last_unstale":"2024-04-25T20:58:45.910363+0000","last_undegraded":"2024-04-25T20:58:45.910363+0000","last_fullsized":"2024-04-25T20:58:45.910363+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.211113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527595+0000","last_change":"2024-04-25T20:56:51.283084+0000","last_active":"2024-04-25T20:57:47.527595+0000","last_peered":"2024-04-25T20:57:47.527595+0000","last_clean":"2024-04-25T20:57:47.527595+0000","last_became_active":"2024-04-25T20:56:51.282692+0000","last_became_peered":"2024-04-25T20:56:51.282692+0000","last_unstale":"2024-04-25T20:57:47.527595+0000","last_undegraded":"2024-04-25T20:57:47.527595+0000","last_fullsized":"2024-04-25T20:57:47.527595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:13:27.527112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64739,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":608,"num_read_kb":375,"num_write":166,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":484239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579949,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1695744,"data_stored":1136458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":484475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":484395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1076716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40484,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704668,"statfs":{"total":95995035648,"available":95953580032,"internally_reserved":0,"allocated":1003520,"data_stored":483864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":544004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":485135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":545745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":484175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":484692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32760,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712392,"statfs":{"total":95995035648,"available":95961489408,"internally_reserved":0,"allocated":1622016,"data_stored":1076009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:58:48.616 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:58:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:50.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:49 smithi092 ceph-mon[32524]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:50.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:49 smithi092 ceph-mon[32524]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:50.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:49 smithi047 ceph-mon[29195]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:50.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:49 smithi047 ceph-mon[29195]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:50.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:49 smithi186 ceph-mon[32529]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:50.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:49 smithi186 ceph-mon[32529]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 -- 172.21.15.47:0/3287529837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34108a70 msgr2=0x7f8a34108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 --2- 172.21.15.47:0/3287529837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34108a70 0x7f8a34108e50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009990 tx=0x7f8a24009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 -- 172.21.15.47:0/3287529837 shutdown_connections 2024-04-25T20:58:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 --2- 172.21.15.47:0/3287529837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34108a70 0x7f8a34108e50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 --2- 172.21.15.47:0/3287529837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a34103480 0x7f8a3410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 --2- 172.21.15.47:0/3287529837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a34102ac0 0x7f8a34102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 -- 172.21.15.47:0/3287529837 >> 172.21.15.47:0/3287529837 conn(0x7f8a34079bb0 msgr2=0x7f8a34079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 -- 172.21.15.47:0/3287529837 shutdown_connections 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.434+0000 7f8a3a761700 1 -- 172.21.15.47:0/3287529837 wait complete. 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.435+0000 7f8a3a761700 1 Processor -- start 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.435+0000 7f8a3a761700 1 -- start start 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a3a761700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a34102ac0 0x7f8a34199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a3a761700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34103480 0x7f8a34199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a3a761700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a34108a70 0x7f8a3419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a3a761700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34112c50 con 0x7f8a34103480 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a3a761700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34112ad0 con 0x7f8a34108a70 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a3a761700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34112dd0 con 0x7f8a34102ac0 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34103480 0x7f8a34199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34103480 0x7f8a34199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48028/0 (socket says 172.21.15.47:48028) 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 -- 172.21.15.47:0/860091950 learned_addr learned my addr 172.21.15.47:0/860091950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a33fff700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a34102ac0 0x7f8a34199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a38cfe700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a34108a70 0x7f8a3419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 -- 172.21.15.47:0/860091950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a34102ac0 msgr2=0x7f8a34199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a34102ac0 0x7f8a34199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 -- 172.21.15.47:0/860091950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a34108a70 msgr2=0x7f8a3419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a34108a70 0x7f8a3419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 -- 172.21.15.47:0/860091950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3419e0c0 con 0x7f8a34103480 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a33fff700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a34102ac0 0x7f8a34199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.436+0000 7f8a337fe700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34103480 0x7f8a34199640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00d950 tx=0x7f8a1c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.437+0000 7f8a317fa700 1 -- 172.21.15.47:0/860091950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c016070 con 0x7f8a34103480 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.437+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3419e3b0 con 0x7f8a34103480 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.437+0000 7f8a317fa700 1 -- 172.21.15.47:0/860091950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c010460 con 0x7f8a34103480 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.437+0000 7f8a317fa700 1 -- 172.21.15.47:0/860091950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00f620 con 0x7f8a34103480 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.437+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a341a5c30 con 0x7f8a34103480 2024-04-25T20:58:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.438+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a34069730 con 0x7f8a34103480 2024-04-25T20:58:50.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.439+0000 7f8a317fa700 1 -- 172.21.15.47:0/860091950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8a1c019070 con 0x7f8a34103480 2024-04-25T20:58:50.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.439+0000 7f8a317fa700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8a200a3c40 0x7f8a200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:50.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.439+0000 7f8a33fff700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8a200a3c40 0x7f8a200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:50.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.439+0000 7f8a317fa700 1 -- 172.21.15.47:0/860091950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f8a1c0cffe0 con 0x7f8a34103480 2024-04-25T20:58:50.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.440+0000 7f8a33fff700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8a200a3c40 0x7f8a200a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a24006010 tx=0x7f8a2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:50.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.444+0000 7f8a317fa700 1 -- 172.21.15.47:0/860091950 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a1c09ca50 con 0x7f8a34103480 2024-04-25T20:58:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.646+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a34062e80 con 0x7f8a200a3c40 2024-04-25T20:58:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.653+0000 7f8a317fa700 1 -- 172.21.15.47:0/860091950 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7f8a34062e80 con 0x7f8a200a3c40 2024-04-25T20:58:50.655 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:58:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8a200a3c40 msgr2=0x7f8a200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8a200a3c40 0x7f8a200a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a24006010 tx=0x7f8a2401a040 comp rx=0 tx=0).stop 2024-04-25T20:58:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34103480 msgr2=0x7f8a34199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34103480 0x7f8a34199640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00d950 tx=0x7f8a1c00dd10 comp rx=0 tx=0).stop 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 shutdown_connections 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f8a200a3c40 0x7f8a200a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a34108a70 0x7f8a3419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a34102ac0 0x7f8a34199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 --2- 172.21.15.47:0/860091950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a34103480 0x7f8a34199640 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 >> 172.21.15.47:0/860091950 conn(0x7f8a34079bb0 msgr2=0x7f8a34101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 shutdown_connections 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:50.656+0000 7f8a3a761700 1 -- 172.21.15.47:0/860091950 wait complete. 2024-04-25T20:58:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-25T20:58:51.313 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-25T20:58:49.947112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655623,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":756,"num_read_kb":496,"num_write":353,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":644,"ondisk_log_size":644,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363016,"kb_used_data":13836,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578808,"statfs":{"total":1151940427776,"available":1151568699392,"internally_reserved":0,"allocated":14168064,"data_stored":7770471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":32,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009262"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.932346+0000","last_change":"2024-04-25T20:56:51.283722+0000","last_active":"2024-04-25T20:58:45.932346+0000","last_peered":"2024-04-25T20:58:45.932346+0000","last_clean":"2024-04-25T20:58:45.932346+0000","last_became_active":"2024-04-25T20:56:51.283359+0000","last_became_peered":"2024-04-25T20:56:51.283359+0000","last_unstale":"2024-04-25T20:58:45.932346+0000","last_undegraded":"2024-04-25T20:58:45.932346+0000","last_fullsized":"2024-04-25T20:58:45.932346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:47.314687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.922366+0000","last_change":"2024-04-25T20:56:51.273199+0000","last_active":"2024-04-25T20:58:45.922366+0000","last_peered":"2024-04-25T20:58:45.922366+0000","last_clean":"2024-04-25T20:58:45.922366+0000","last_became_active":"2024-04-25T20:56:51.272838+0000","last_became_peered":"2024-04-25T20:56:51.272838+0000","last_unstale":"2024-04-25T20:58:45.922366+0000","last_undegraded":"2024-04-25T20:58:45.922366+0000","last_fullsized":"2024-04-25T20:58:45.922366+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:58.033315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.924606+0000","last_change":"2024-04-25T20:56:51.281461+0000","last_active":"2024-04-25T20:58:45.924606+0000","last_peered":"2024-04-25T20:58:45.924606+0000","last_clean":"2024-04-25T20:58:45.924606+0000","last_became_active":"2024-04-25T20:56:51.281141+0000","last_became_peered":"2024-04-25T20:56:51.281141+0000","last_unstale":"2024-04-25T20:58:45.924606+0000","last_undegraded":"2024-04-25T20:58:45.924606+0000","last_fullsized":"2024-04-25T20:58:45.924606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.914458+0000","last_change":"2024-04-25T20:56:51.283187+0000","last_active":"2024-04-25T20:58:45.914458+0000","last_peered":"2024-04-25T20:58:45.914458+0000","last_clean":"2024-04-25T20:58:45.914458+0000","last_became_active":"2024-04-25T20:56:51.282640+0000","last_became_peered":"2024-04-25T20:56:51.282640+0000","last_unstale":"2024-04-25T20:58:45.914458+0000","last_undegraded":"2024-04-25T20:58:45.914458+0000","last_fullsized":"2024-04-25T20:58:45.914458+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:15.588694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.919912+0000","last_change":"2024-04-25T20:56:51.273204+0000","last_active":"2024-04-25T20:58:45.919912+0000","last_peered":"2024-04-25T20:58:45.919912+0000","last_clean":"2024-04-25T20:58:45.919912+0000","last_became_active":"2024-04-25T20:56:51.272126+0000","last_became_peered":"2024-04-25T20:56:51.272126+0000","last_unstale":"2024-04-25T20:58:45.919912+0000","last_undegraded":"2024-04-25T20:58:45.919912+0000","last_fullsized":"2024-04-25T20:58:45.919912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:31.684294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793774+0000","last_change":"2024-04-25T20:56:51.270762+0000","last_active":"2024-04-25T20:57:47.793774+0000","last_peered":"2024-04-25T20:57:47.793774+0000","last_clean":"2024-04-25T20:57:47.793774+0000","last_became_active":"2024-04-25T20:56:51.270444+0000","last_became_peered":"2024-04-25T20:56:51.270444+0000","last_unstale":"2024-04-25T20:57:47.793774+0000","last_undegraded":"2024-04-25T20:57:47.793774+0000","last_fullsized":"2024-04-25T20:57:47.793774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:00.155417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.909882+0000","last_change":"2024-04-25T20:56:51.276029+0000","last_active":"2024-04-25T20:58:45.909882+0000","last_peered":"2024-04-25T20:58:45.909882+0000","last_clean":"2024-04-25T20:58:45.909882+0000","last_became_active":"2024-04-25T20:56:51.275272+0000","last_became_peered":"2024-04-25T20:56:51.275272+0000","last_unstale":"2024-04-25T20:58:45.909882+0000","last_undegraded":"2024-04-25T20:58:45.909882+0000","last_fullsized":"2024-04-25T20:58:45.909882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.428444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.916618+0000","last_change":"2024-04-25T20:56:51.273769+0000","last_active":"2024-04-25T20:58:45.916618+0000","last_peered":"2024-04-25T20:58:45.916618+0000","last_clean":"2024-04-25T20:58:45.916618+0000","last_became_active":"2024-04-25T20:56:51.273412+0000","last_became_peered":"2024-04-25T20:56:51.273412+0000","last_unstale":"2024-04-25T20:58:45.916618+0000","last_undegraded":"2024-04-25T20:58:45.916618+0000","last_fullsized":"2024-04-25T20:58:45.916618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.324332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299708+0000","last_change":"2024-04-25T20:56:51.277097+0000","last_active":"2024-04-25T20:57:48.299708+0000","last_peered":"2024-04-25T20:57:48.299708+0000","last_clean":"2024-04-25T20:57:48.299708+0000","last_became_active":"2024-04-25T20:56:51.276768+0000","last_became_peered":"2024-04-25T20:56:51.276768+0000","last_unstale":"2024-04-25T20:57:48.299708+0000","last_undegraded":"2024-04-25T20:57:48.299708+0000","last_fullsized":"2024-04-25T20:57:48.299708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:23.276470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.794116+0000","last_change":"2024-04-25T20:56:51.279046+0000","last_active":"2024-04-25T20:57:47.794116+0000","last_peered":"2024-04-25T20:57:47.794116+0000","last_clean":"2024-04-25T20:57:47.794116+0000","last_became_active":"2024-04-25T20:56:51.278701+0000","last_became_peered":"2024-04-25T20:56:51.278701+0000","last_unstale":"2024-04-25T20:57:47.794116+0000","last_undegraded":"2024-04-25T20:57:47.794116+0000","last_fullsized":"2024-04-25T20:57:47.794116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.370482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.932536+0000","last_change":"2024-04-25T20:56:51.271593+0000","last_active":"2024-04-25T20:58:45.932536+0000","last_peered":"2024-04-25T20:58:45.932536+0000","last_clean":"2024-04-25T20:58:45.932536+0000","last_became_active":"2024-04-25T20:56:51.271224+0000","last_became_peered":"2024-04-25T20:56:51.271224+0000","last_unstale":"2024-04-25T20:58:45.932536+0000","last_undegraded":"2024-04-25T20:58:45.932536+0000","last_fullsized":"2024-04-25T20:58:45.932536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:46.847929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.914572+0000","last_change":"2024-04-25T20:56:51.278281+0000","last_active":"2024-04-25T20:58:45.914572+0000","last_peered":"2024-04-25T20:58:45.914572+0000","last_clean":"2024-04-25T20:58:45.914572+0000","last_became_active":"2024-04-25T20:56:51.277947+0000","last_became_peered":"2024-04-25T20:56:51.277947+0000","last_unstale":"2024-04-25T20:58:45.914572+0000","last_undegraded":"2024-04-25T20:58:45.914572+0000","last_fullsized":"2024-04-25T20:58:45.914572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:02.509296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.915415+0000","last_change":"2024-04-25T20:56:51.273784+0000","last_active":"2024-04-25T20:58:45.915415+0000","last_peered":"2024-04-25T20:58:45.915415+0000","last_clean":"2024-04-25T20:58:45.915415+0000","last_became_active":"2024-04-25T20:56:51.273489+0000","last_became_peered":"2024-04-25T20:56:51.273489+0000","last_unstale":"2024-04-25T20:58:45.915415+0000","last_undegraded":"2024-04-25T20:58:45.915415+0000","last_fullsized":"2024-04-25T20:58:45.915415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:00.765186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.925980+0000","last_change":"2024-04-25T20:56:51.269829+0000","last_active":"2024-04-25T20:58:45.925980+0000","last_peered":"2024-04-25T20:58:45.925980+0000","last_clean":"2024-04-25T20:58:45.925980+0000","last_became_active":"2024-04-25T20:56:51.269446+0000","last_became_peered":"2024-04-25T20:56:51.269446+0000","last_unstale":"2024-04-25T20:58:45.925980+0000","last_undegraded":"2024-04-25T20:58:45.925980+0000","last_fullsized":"2024-04-25T20:58:45.925980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:12.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.926062+0000","last_change":"2024-04-25T20:56:51.278117+0000","last_active":"2024-04-25T20:58:45.926062+0000","last_peered":"2024-04-25T20:58:45.926062+0000","last_clean":"2024-04-25T20:58:45.926062+0000","last_became_active":"2024-04-25T20:56:51.277596+0000","last_became_peered":"2024-04-25T20:56:51.277596+0000","last_unstale":"2024-04-25T20:58:45.926062+0000","last_undegraded":"2024-04-25T20:58:45.926062+0000","last_fullsized":"2024-04-25T20:58:45.926062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:28.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299349+0000","last_change":"2024-04-25T20:56:51.281575+0000","last_active":"2024-04-25T20:57:48.299349+0000","last_peered":"2024-04-25T20:57:48.299349+0000","last_clean":"2024-04-25T20:57:48.299349+0000","last_became_active":"2024-04-25T20:56:51.281217+0000","last_became_peered":"2024-04-25T20:56:51.281217+0000","last_unstale":"2024-04-25T20:57:48.299349+0000","last_undegraded":"2024-04-25T20:57:48.299349+0000","last_fullsized":"2024-04-25T20:57:48.299349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:05.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.932632+0000","last_change":"2024-04-25T20:56:51.284608+0000","last_active":"2024-04-25T20:58:45.932632+0000","last_peered":"2024-04-25T20:58:45.932632+0000","last_clean":"2024-04-25T20:58:45.932632+0000","last_became_active":"2024-04-25T20:56:51.284257+0000","last_became_peered":"2024-04-25T20:56:51.284257+0000","last_unstale":"2024-04-25T20:58:45.932632+0000","last_undegraded":"2024-04-25T20:58:45.932632+0000","last_fullsized":"2024-04-25T20:58:45.932632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:44:59.184192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.930291+0000","last_change":"2024-04-25T20:56:51.276770+0000","last_active":"2024-04-25T20:58:45.930291+0000","last_peered":"2024-04-25T20:58:45.930291+0000","last_clean":"2024-04-25T20:58:45.930291+0000","last_became_active":"2024-04-25T20:56:51.276431+0000","last_became_peered":"2024-04-25T20:56:51.276431+0000","last_unstale":"2024-04-25T20:58:45.930291+0000","last_undegraded":"2024-04-25T20:58:45.930291+0000","last_fullsized":"2024-04-25T20:58:45.930291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:14:25.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524537+0000","last_change":"2024-04-25T20:56:51.283728+0000","last_active":"2024-04-25T20:57:47.524537+0000","last_peered":"2024-04-25T20:57:47.524537+0000","last_clean":"2024-04-25T20:57:47.524537+0000","last_became_active":"2024-04-25T20:56:51.283370+0000","last_became_peered":"2024-04-25T20:56:51.283370+0000","last_unstale":"2024-04-25T20:57:47.524537+0000","last_undegraded":"2024-04-25T20:57:47.524537+0000","last_fullsized":"2024-04-25T20:57:47.524537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:18.042769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.523509+0000","last_change":"2024-04-25T20:56:48.259737+0000","last_active":"2024-04-25T20:57:47.523509+0000","last_peered":"2024-04-25T20:57:47.523509+0000","last_clean":"2024-04-25T20:57:47.523509+0000","last_became_active":"2024-04-25T20:56:48.259442+0000","last_became_peered":"2024-04-25T20:56:48.259442+0000","last_unstale":"2024-04-25T20:57:47.523509+0000","last_undegraded":"2024-04-25T20:57:47.523509+0000","last_fullsized":"2024-04-25T20:57:47.523509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:23.191536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.911865+0000","last_change":"2024-04-25T20:56:51.274510+0000","last_active":"2024-04-25T20:58:45.911865+0000","last_peered":"2024-04-25T20:58:45.911865+0000","last_clean":"2024-04-25T20:58:45.911865+0000","last_became_active":"2024-04-25T20:56:51.274137+0000","last_became_peered":"2024-04-25T20:56:51.274137+0000","last_unstale":"2024-04-25T20:58:45.911865+0000","last_undegraded":"2024-04-25T20:58:45.911865+0000","last_fullsized":"2024-04-25T20:58:45.911865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:05.903617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526710+0000","last_change":"2024-04-25T20:56:48.947850+0000","last_active":"2024-04-25T20:57:47.526710+0000","last_peered":"2024-04-25T20:57:47.526710+0000","last_clean":"2024-04-25T20:57:47.526710+0000","last_became_active":"2024-04-25T20:56:48.946160+0000","last_became_peered":"2024-04-25T20:56:48.946160+0000","last_unstale":"2024-04-25T20:57:47.526710+0000","last_undegraded":"2024-04-25T20:57:47.526710+0000","last_fullsized":"2024-04-25T20:57:47.526710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:32.692390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.927234+0000","last_change":"2024-04-25T20:56:51.280478+0000","last_active":"2024-04-25T20:58:45.927234+0000","last_peered":"2024-04-25T20:58:45.927234+0000","last_clean":"2024-04-25T20:58:45.927234+0000","last_became_active":"2024-04-25T20:56:51.280173+0000","last_became_peered":"2024-04-25T20:56:51.280173+0000","last_unstale":"2024-04-25T20:58:45.927234+0000","last_undegraded":"2024-04-25T20:58:45.927234+0000","last_fullsized":"2024-04-25T20:58:45.927234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:48.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.535264+0000","last_change":"2024-04-25T20:57:47.535264+0000","last_active":"2024-04-25T20:57:47.535264+0000","last_peered":"2024-04-25T20:57:47.535264+0000","last_clean":"2024-04-25T20:57:47.535264+0000","last_became_active":"2024-04-25T20:57:47.531152+0000","last_became_peered":"2024-04-25T20:57:47.531152+0000","last_unstale":"2024-04-25T20:57:47.535264+0000","last_undegraded":"2024-04-25T20:57:47.535264+0000","last_fullsized":"2024-04-25T20:57:47.535264+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:19.085455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299473+0000","last_change":"2024-04-25T20:56:51.278383+0000","last_active":"2024-04-25T20:57:48.299473+0000","last_peered":"2024-04-25T20:57:48.299473+0000","last_clean":"2024-04-25T20:57:48.299473+0000","last_became_active":"2024-04-25T20:56:51.277603+0000","last_became_peered":"2024-04-25T20:56:51.277603+0000","last_unstale":"2024-04-25T20:57:48.299473+0000","last_undegraded":"2024-04-25T20:57:48.299473+0000","last_fullsized":"2024-04-25T20:57:48.299473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:12.197940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527024+0000","last_change":"2024-04-25T20:56:48.258869+0000","last_active":"2024-04-25T20:57:47.527024+0000","last_peered":"2024-04-25T20:57:47.527024+0000","last_clean":"2024-04-25T20:57:47.527024+0000","last_became_active":"2024-04-25T20:56:48.258414+0000","last_became_peered":"2024-04-25T20:56:48.258414+0000","last_unstale":"2024-04-25T20:57:47.527024+0000","last_undegraded":"2024-04-25T20:57:47.527024+0000","last_fullsized":"2024-04-25T20:57:47.527024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:09.889936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.932309+0000","last_change":"2024-04-25T20:57:48.300637+0000","last_active":"2024-04-25T20:58:45.932309+0000","last_peered":"2024-04-25T20:58:45.932309+0000","last_clean":"2024-04-25T20:58:45.932309+0000","last_became_active":"2024-04-25T20:57:48.300285+0000","last_became_peered":"2024-04-25T20:57:48.300285+0000","last_unstale":"2024-04-25T20:58:45.932309+0000","last_undegraded":"2024-04-25T20:58:45.932309+0000","last_fullsized":"2024-04-25T20:58:45.932309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:31.810671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":136,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:46.890822+0000","last_change":"2024-04-25T20:56:48.262543+0000","last_active":"2024-04-25T20:58:46.890822+0000","last_peered":"2024-04-25T20:58:46.890822+0000","last_clean":"2024-04-25T20:58:46.890822+0000","last_became_active":"2024-04-25T20:56:48.262086+0000","last_became_peered":"2024-04-25T20:56:48.262086+0000","last_unstale":"2024-04-25T20:58:46.890822+0000","last_undegraded":"2024-04-25T20:58:46.890822+0000","last_fullsized":"2024-04-25T20:58:46.890822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:43.722131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.923691+0000","last_change":"2024-04-25T20:56:51.273500+0000","last_active":"2024-04-25T20:58:45.923691+0000","last_peered":"2024-04-25T20:58:45.923691+0000","last_clean":"2024-04-25T20:58:45.923691+0000","last_became_active":"2024-04-25T20:56:51.272882+0000","last_became_peered":"2024-04-25T20:56:51.272882+0000","last_unstale":"2024-04-25T20:58:45.923691+0000","last_undegraded":"2024-04-25T20:58:45.923691+0000","last_fullsized":"2024-04-25T20:58:45.923691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:05.682258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793722+0000","last_change":"2024-04-25T20:56:48.255225+0000","last_active":"2024-04-25T20:57:47.793722+0000","last_peered":"2024-04-25T20:57:47.793722+0000","last_clean":"2024-04-25T20:57:47.793722+0000","last_became_active":"2024-04-25T20:56:48.254644+0000","last_became_peered":"2024-04-25T20:56:48.254644+0000","last_unstale":"2024-04-25T20:57:47.793722+0000","last_undegraded":"2024-04-25T20:57:47.793722+0000","last_fullsized":"2024-04-25T20:57:47.793722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:35.724612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.907442+0000","last_change":"2024-04-25T20:56:51.270709+0000","last_active":"2024-04-25T20:58:45.907442+0000","last_peered":"2024-04-25T20:58:45.907442+0000","last_clean":"2024-04-25T20:58:45.907442+0000","last_became_active":"2024-04-25T20:56:51.270295+0000","last_became_peered":"2024-04-25T20:56:51.270295+0000","last_unstale":"2024-04-25T20:58:45.907442+0000","last_undegraded":"2024-04-25T20:58:45.907442+0000","last_fullsized":"2024-04-25T20:58:45.907442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:41.998953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793738+0000","last_change":"2024-04-25T20:56:48.256498+0000","last_active":"2024-04-25T20:57:47.793738+0000","last_peered":"2024-04-25T20:57:47.793738+0000","last_clean":"2024-04-25T20:57:47.793738+0000","last_became_active":"2024-04-25T20:56:48.256140+0000","last_became_peered":"2024-04-25T20:56:48.256140+0000","last_unstale":"2024-04-25T20:57:47.793738+0000","last_undegraded":"2024-04-25T20:57:47.793738+0000","last_fullsized":"2024-04-25T20:57:47.793738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:41.458235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.922863+0000","last_change":"2024-04-25T20:56:51.267575+0000","last_active":"2024-04-25T20:58:45.922863+0000","last_peered":"2024-04-25T20:58:45.922863+0000","last_clean":"2024-04-25T20:58:45.922863+0000","last_became_active":"2024-04-25T20:56:51.267265+0000","last_became_peered":"2024-04-25T20:56:51.267265+0000","last_unstale":"2024-04-25T20:58:45.922863+0000","last_undegraded":"2024-04-25T20:58:45.922863+0000","last_fullsized":"2024-04-25T20:58:45.922863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:30.760555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591411+0000","last_change":"2024-04-25T20:56:48.260747+0000","last_active":"2024-04-25T20:57:47.591411+0000","last_peered":"2024-04-25T20:57:47.591411+0000","last_clean":"2024-04-25T20:57:47.591411+0000","last_became_active":"2024-04-25T20:56:48.260339+0000","last_became_peered":"2024-04-25T20:56:48.260339+0000","last_unstale":"2024-04-25T20:57:47.591411+0000","last_undegraded":"2024-04-25T20:57:47.591411+0000","last_fullsized":"2024-04-25T20:57:47.591411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:19.769631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.923200+0000","last_change":"2024-04-25T20:57:47.795125+0000","last_active":"2024-04-25T20:58:45.923200+0000","last_peered":"2024-04-25T20:58:45.923200+0000","last_clean":"2024-04-25T20:58:45.923200+0000","last_became_active":"2024-04-25T20:57:47.794642+0000","last_became_peered":"2024-04-25T20:57:47.794642+0000","last_unstale":"2024-04-25T20:58:45.923200+0000","last_undegraded":"2024-04-25T20:58:45.923200+0000","last_fullsized":"2024-04-25T20:58:45.923200+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:15.443731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:43.879294+0000","last_change":"2024-04-25T20:56:48.256113+0000","last_active":"2024-04-25T20:58:43.879294+0000","last_peered":"2024-04-25T20:58:43.879294+0000","last_clean":"2024-04-25T20:58:43.879294+0000","last_became_active":"2024-04-25T20:56:48.255644+0000","last_became_peered":"2024-04-25T20:56:48.255644+0000","last_unstale":"2024-04-25T20:58:43.879294+0000","last_undegraded":"2024-04-25T20:58:43.879294+0000","last_fullsized":"2024-04-25T20:58:43.879294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:12.100099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299841+0000","last_change":"2024-04-25T20:56:51.276990+0000","last_active":"2024-04-25T20:57:48.299841+0000","last_peered":"2024-04-25T20:57:48.299841+0000","last_clean":"2024-04-25T20:57:48.299841+0000","last_became_active":"2024-04-25T20:56:51.276638+0000","last_became_peered":"2024-04-25T20:56:51.276638+0000","last_unstale":"2024-04-25T20:57:48.299841+0000","last_undegraded":"2024-04-25T20:57:48.299841+0000","last_fullsized":"2024-04-25T20:57:48.299841+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:06.645278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531382+0000","last_change":"2024-04-25T20:56:48.263216+0000","last_active":"2024-04-25T20:57:47.531382+0000","last_peered":"2024-04-25T20:57:47.531382+0000","last_clean":"2024-04-25T20:57:47.531382+0000","last_became_active":"2024-04-25T20:56:48.262854+0000","last_became_peered":"2024-04-25T20:56:48.262854+0000","last_unstale":"2024-04-25T20:57:47.531382+0000","last_undegraded":"2024-04-25T20:57:47.531382+0000","last_fullsized":"2024-04-25T20:57:47.531382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:00.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.927321+0000","last_change":"2024-04-25T20:56:51.278479+0000","last_active":"2024-04-25T20:58:45.927321+0000","last_peered":"2024-04-25T20:58:45.927321+0000","last_clean":"2024-04-25T20:58:45.927321+0000","last_became_active":"2024-04-25T20:56:51.276285+0000","last_became_peered":"2024-04-25T20:56:51.276285+0000","last_unstale":"2024-04-25T20:58:45.927321+0000","last_undegraded":"2024-04-25T20:58:45.927321+0000","last_fullsized":"2024-04-25T20:58:45.927321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:14.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591683+0000","last_change":"2024-04-25T20:56:48.946808+0000","last_active":"2024-04-25T20:57:47.591683+0000","last_peered":"2024-04-25T20:57:47.591683+0000","last_clean":"2024-04-25T20:57:47.591683+0000","last_became_active":"2024-04-25T20:56:48.946421+0000","last_became_peered":"2024-04-25T20:56:48.946421+0000","last_unstale":"2024-04-25T20:57:47.591683+0000","last_undegraded":"2024-04-25T20:57:47.591683+0000","last_fullsized":"2024-04-25T20:57:47.591683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:47.763950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299666+0000","last_change":"2024-04-25T20:56:51.274761+0000","last_active":"2024-04-25T20:57:48.299666+0000","last_peered":"2024-04-25T20:57:48.299666+0000","last_clean":"2024-04-25T20:57:48.299666+0000","last_became_active":"2024-04-25T20:56:51.274452+0000","last_became_peered":"2024-04-25T20:56:51.274452+0000","last_unstale":"2024-04-25T20:57:48.299666+0000","last_undegraded":"2024-04-25T20:57:48.299666+0000","last_fullsized":"2024-04-25T20:57:48.299666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:32.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'28","reported_seq":257,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:44.880941+0000","last_change":"2024-04-25T20:56:48.258982+0000","last_active":"2024-04-25T20:58:44.880941+0000","last_peered":"2024-04-25T20:58:44.880941+0000","last_clean":"2024-04-25T20:58:44.880941+0000","last_became_active":"2024-04-25T20:56:48.258593+0000","last_became_peered":"2024-04-25T20:56:48.258593+0000","last_unstale":"2024-04-25T20:58:44.880941+0000","last_undegraded":"2024-04-25T20:58:44.880941+0000","last_fullsized":"2024-04-25T20:58:44.880941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:20.518202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":203,"num_read_kb":160,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.906516+0000","last_change":"2024-04-25T20:56:51.276361+0000","last_active":"2024-04-25T20:58:45.906516+0000","last_peered":"2024-04-25T20:58:45.906516+0000","last_clean":"2024-04-25T20:58:45.906516+0000","last_became_active":"2024-04-25T20:56:51.276045+0000","last_became_peered":"2024-04-25T20:56:51.276045+0000","last_unstale":"2024-04-25T20:58:45.906516+0000","last_undegraded":"2024-04-25T20:58:45.906516+0000","last_fullsized":"2024-04-25T20:58:45.906516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:04.325068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524318+0000","last_change":"2024-04-25T20:56:48.252358+0000","last_active":"2024-04-25T20:57:47.524318+0000","last_peered":"2024-04-25T20:57:47.524318+0000","last_clean":"2024-04-25T20:57:47.524318+0000","last_became_active":"2024-04-25T20:56:48.252027+0000","last_became_peered":"2024-04-25T20:56:48.252027+0000","last_unstale":"2024-04-25T20:57:47.524318+0000","last_undegraded":"2024-04-25T20:57:47.524318+0000","last_fullsized":"2024-04-25T20:57:47.524318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:03.024487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.910850+0000","last_change":"2024-04-25T20:56:51.279599+0000","last_active":"2024-04-25T20:58:45.910850+0000","last_peered":"2024-04-25T20:58:45.910850+0000","last_clean":"2024-04-25T20:58:45.910850+0000","last_became_active":"2024-04-25T20:56:51.279294+0000","last_became_peered":"2024-04-25T20:56:51.279294+0000","last_unstale":"2024-04-25T20:58:45.910850+0000","last_undegraded":"2024-04-25T20:58:45.910850+0000","last_fullsized":"2024-04-25T20:58:45.910850+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591497+0000","last_change":"2024-04-25T20:56:48.261005+0000","last_active":"2024-04-25T20:57:47.591497+0000","last_peered":"2024-04-25T20:57:47.591497+0000","last_clean":"2024-04-25T20:57:47.591497+0000","last_became_active":"2024-04-25T20:56:48.260543+0000","last_became_peered":"2024-04-25T20:56:48.260543+0000","last_unstale":"2024-04-25T20:57:47.591497+0000","last_undegraded":"2024-04-25T20:57:47.591497+0000","last_fullsized":"2024-04-25T20:57:47.591497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:41:54.326036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.918708+0000","last_change":"2024-04-25T20:56:51.271753+0000","last_active":"2024-04-25T20:58:45.918708+0000","last_peered":"2024-04-25T20:58:45.918708+0000","last_clean":"2024-04-25T20:58:45.918708+0000","last_became_active":"2024-04-25T20:56:51.271392+0000","last_became_peered":"2024-04-25T20:56:51.271392+0000","last_unstale":"2024-04-25T20:58:45.918708+0000","last_undegraded":"2024-04-25T20:58:45.918708+0000","last_fullsized":"2024-04-25T20:58:45.918708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:26.605647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:46.921214+0000","last_change":"2024-04-25T20:56:48.260729+0000","last_active":"2024-04-25T20:58:46.921214+0000","last_peered":"2024-04-25T20:58:46.921214+0000","last_clean":"2024-04-25T20:58:46.921214+0000","last_became_active":"2024-04-25T20:56:48.260377+0000","last_became_peered":"2024-04-25T20:56:48.260377+0000","last_unstale":"2024-04-25T20:58:46.921214+0000","last_undegraded":"2024-04-25T20:58:46.921214+0000","last_fullsized":"2024-04-25T20:58:46.921214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:21.582266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.910786+0000","last_change":"2024-04-25T20:56:51.275691+0000","last_active":"2024-04-25T20:58:45.910786+0000","last_peered":"2024-04-25T20:58:45.910786+0000","last_clean":"2024-04-25T20:58:45.910786+0000","last_became_active":"2024-04-25T20:56:51.275220+0000","last_became_peered":"2024-04-25T20:56:51.275220+0000","last_unstale":"2024-04-25T20:58:45.910786+0000","last_undegraded":"2024-04-25T20:58:45.910786+0000","last_fullsized":"2024-04-25T20:58:45.910786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:24.367413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526402+0000","last_change":"2024-04-25T20:56:48.946465+0000","last_active":"2024-04-25T20:57:47.526402+0000","last_peered":"2024-04-25T20:57:47.526402+0000","last_clean":"2024-04-25T20:57:47.526402+0000","last_became_active":"2024-04-25T20:56:48.946093+0000","last_became_peered":"2024-04-25T20:56:48.946093+0000","last_unstale":"2024-04-25T20:57:47.526402+0000","last_undegraded":"2024-04-25T20:57:47.526402+0000","last_fullsized":"2024-04-25T20:57:47.526402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:29.502073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.912184+0000","last_change":"2024-04-25T20:56:51.278464+0000","last_active":"2024-04-25T20:58:45.912184+0000","last_peered":"2024-04-25T20:58:45.912184+0000","last_clean":"2024-04-25T20:58:45.912184+0000","last_became_active":"2024-04-25T20:56:51.278119+0000","last_became_peered":"2024-04-25T20:56:51.278119+0000","last_unstale":"2024-04-25T20:58:45.912184+0000","last_undegraded":"2024-04-25T20:58:45.912184+0000","last_fullsized":"2024-04-25T20:58:45.912184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:57:37.224817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.528952+0000","last_change":"2024-04-25T20:56:48.255123+0000","last_active":"2024-04-25T20:57:47.528952+0000","last_peered":"2024-04-25T20:57:47.528952+0000","last_clean":"2024-04-25T20:57:47.528952+0000","last_became_active":"2024-04-25T20:56:48.254680+0000","last_became_peered":"2024-04-25T20:56:48.254680+0000","last_unstale":"2024-04-25T20:57:47.528952+0000","last_undegraded":"2024-04-25T20:57:47.528952+0000","last_fullsized":"2024-04-25T20:57:47.528952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:24.089590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.921457+0000","last_change":"2024-04-25T20:57:47.592950+0000","last_active":"2024-04-25T20:58:45.921457+0000","last_peered":"2024-04-25T20:58:45.921457+0000","last_clean":"2024-04-25T20:58:45.921457+0000","last_became_active":"2024-04-25T20:57:47.592486+0000","last_became_peered":"2024-04-25T20:57:47.592486+0000","last_unstale":"2024-04-25T20:58:45.921457+0000","last_undegraded":"2024-04-25T20:58:45.921457+0000","last_fullsized":"2024-04-25T20:58:45.921457+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:43.218061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.523705+0000","last_change":"2024-04-25T20:56:48.947508+0000","last_active":"2024-04-25T20:57:47.523705+0000","last_peered":"2024-04-25T20:57:47.523705+0000","last_clean":"2024-04-25T20:57:47.523705+0000","last_became_active":"2024-04-25T20:56:48.946582+0000","last_became_peered":"2024-04-25T20:56:48.946582+0000","last_unstale":"2024-04-25T20:57:47.523705+0000","last_undegraded":"2024-04-25T20:57:47.523705+0000","last_fullsized":"2024-04-25T20:57:47.523705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:11.198667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793925+0000","last_change":"2024-04-25T20:56:51.273560+0000","last_active":"2024-04-25T20:57:47.793925+0000","last_peered":"2024-04-25T20:57:47.793925+0000","last_clean":"2024-04-25T20:57:47.793925+0000","last_became_active":"2024-04-25T20:56:51.273182+0000","last_became_peered":"2024-04-25T20:56:51.273182+0000","last_unstale":"2024-04-25T20:57:47.793925+0000","last_undegraded":"2024-04-25T20:57:47.793925+0000","last_fullsized":"2024-04-25T20:57:47.793925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:47.012771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531232+0000","last_change":"2024-04-25T20:56:48.260754+0000","last_active":"2024-04-25T20:57:47.531232+0000","last_peered":"2024-04-25T20:57:47.531232+0000","last_clean":"2024-04-25T20:57:47.531232+0000","last_became_active":"2024-04-25T20:56:48.260420+0000","last_became_peered":"2024-04-25T20:56:48.260420+0000","last_unstale":"2024-04-25T20:57:47.531232+0000","last_undegraded":"2024-04-25T20:57:47.531232+0000","last_fullsized":"2024-04-25T20:57:47.531232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:00.219023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793815+0000","last_change":"2024-04-25T20:56:51.271944+0000","last_active":"2024-04-25T20:57:47.793815+0000","last_peered":"2024-04-25T20:57:47.793815+0000","last_clean":"2024-04-25T20:57:47.793815+0000","last_became_active":"2024-04-25T20:56:51.270656+0000","last_became_peered":"2024-04-25T20:56:51.270656+0000","last_unstale":"2024-04-25T20:57:47.793815+0000","last_undegraded":"2024-04-25T20:57:47.793815+0000","last_fullsized":"2024-04-25T20:57:47.793815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:00.624888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.920506+0000","last_change":"2024-04-25T20:57:47.526172+0000","last_active":"2024-04-25T20:58:45.920506+0000","last_peered":"2024-04-25T20:58:45.920506+0000","last_clean":"2024-04-25T20:58:45.920506+0000","last_became_active":"2024-04-25T20:57:47.525820+0000","last_became_peered":"2024-04-25T20:57:47.525820+0000","last_unstale":"2024-04-25T20:58:45.920506+0000","last_undegraded":"2024-04-25T20:58:45.920506+0000","last_fullsized":"2024-04-25T20:58:45.920506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:56.458959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":129,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.878953+0000","last_change":"2024-04-25T20:56:48.946471+0000","last_active":"2024-04-25T20:58:45.878953+0000","last_peered":"2024-04-25T20:58:45.878953+0000","last_clean":"2024-04-25T20:58:45.878953+0000","last_became_active":"2024-04-25T20:56:48.946029+0000","last_became_peered":"2024-04-25T20:56:48.946029+0000","last_unstale":"2024-04-25T20:58:45.878953+0000","last_undegraded":"2024-04-25T20:58:45.878953+0000","last_fullsized":"2024-04-25T20:58:45.878953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:30:16.400157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.929052+0000","last_change":"2024-04-25T20:57:48.300758+0000","last_active":"2024-04-25T20:58:45.929052+0000","last_peered":"2024-04-25T20:58:45.929052+0000","last_clean":"2024-04-25T20:58:45.929052+0000","last_became_active":"2024-04-25T20:57:48.300328+0000","last_became_peered":"2024-04-25T20:57:48.300328+0000","last_unstale":"2024-04-25T20:58:45.929052+0000","last_undegraded":"2024-04-25T20:58:45.929052+0000","last_fullsized":"2024-04-25T20:58:45.929052+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:31.931834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524170+0000","last_change":"2024-04-25T20:56:48.258479+0000","last_active":"2024-04-25T20:57:47.524170+0000","last_peered":"2024-04-25T20:57:47.524170+0000","last_clean":"2024-04-25T20:57:47.524170+0000","last_became_active":"2024-04-25T20:56:48.257951+0000","last_became_peered":"2024-04-25T20:56:48.257951+0000","last_unstale":"2024-04-25T20:57:47.524170+0000","last_undegraded":"2024-04-25T20:57:47.524170+0000","last_fullsized":"2024-04-25T20:57:47.524170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:13.812201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.530900+0000","last_change":"2024-04-25T20:55:28.701357+0000","last_active":"2024-04-25T20:57:47.530900+0000","last_peered":"2024-04-25T20:57:47.530900+0000","last_clean":"2024-04-25T20:57:47.530900+0000","last_became_active":"2024-04-25T20:55:28.700906+0000","last_became_peered":"2024-04-25T20:55:28.700906+0000","last_unstale":"2024-04-25T20:57:47.530900+0000","last_undegraded":"2024-04-25T20:57:47.530900+0000","last_fullsized":"2024-04-25T20:57:47.530900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_clean_scrub_stamp":"2024-04-25T20:52:10.334323+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:37:35.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.907807+0000","last_change":"2024-04-25T20:56:51.275249+0000","last_active":"2024-04-25T20:58:45.907807+0000","last_peered":"2024-04-25T20:58:45.907807+0000","last_clean":"2024-04-25T20:58:45.907807+0000","last_became_active":"2024-04-25T20:56:51.274938+0000","last_became_peered":"2024-04-25T20:56:51.274938+0000","last_unstale":"2024-04-25T20:58:45.907807+0000","last_undegraded":"2024-04-25T20:58:45.907807+0000","last_fullsized":"2024-04-25T20:58:45.907807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:53.933324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.531104+0000","last_change":"2024-04-25T20:56:48.259357+0000","last_active":"2024-04-25T20:57:47.531104+0000","last_peered":"2024-04-25T20:57:47.531104+0000","last_clean":"2024-04-25T20:57:47.531104+0000","last_became_active":"2024-04-25T20:56:48.258840+0000","last_became_peered":"2024-04-25T20:56:48.258840+0000","last_unstale":"2024-04-25T20:57:47.531104+0000","last_undegraded":"2024-04-25T20:57:47.531104+0000","last_fullsized":"2024-04-25T20:57:47.531104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:30.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.527510+0000","last_change":"2024-04-25T20:56:48.946498+0000","last_active":"2024-04-25T20:57:47.527510+0000","last_peered":"2024-04-25T20:57:47.527510+0000","last_clean":"2024-04-25T20:57:47.527510+0000","last_became_active":"2024-04-25T20:56:48.946124+0000","last_became_peered":"2024-04-25T20:56:48.946124+0000","last_unstale":"2024-04-25T20:57:47.527510+0000","last_undegraded":"2024-04-25T20:57:47.527510+0000","last_fullsized":"2024-04-25T20:57:47.527510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:08.545574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.908819+0000","last_change":"2024-04-25T20:56:51.276093+0000","last_active":"2024-04-25T20:58:45.908819+0000","last_peered":"2024-04-25T20:58:45.908819+0000","last_clean":"2024-04-25T20:58:45.908819+0000","last_became_active":"2024-04-25T20:56:51.275771+0000","last_became_peered":"2024-04-25T20:56:51.275771+0000","last_unstale":"2024-04-25T20:58:45.908819+0000","last_undegraded":"2024-04-25T20:58:45.908819+0000","last_fullsized":"2024-04-25T20:58:45.908819+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:15.159208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.533696+0000","last_change":"2024-04-25T20:57:47.533338+0000","last_active":"2024-04-25T20:57:47.533696+0000","last_peered":"2024-04-25T20:57:47.533696+0000","last_clean":"2024-04-25T20:57:47.533696+0000","last_became_active":"2024-04-25T20:57:47.529539+0000","last_became_peered":"2024-04-25T20:57:47.529539+0000","last_unstale":"2024-04-25T20:57:47.533696+0000","last_undegraded":"2024-04-25T20:57:47.533696+0000","last_fullsized":"2024-04-25T20:57:47.533696+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:25.099128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.923091+0000","last_change":"2024-04-25T20:56:51.275461+0000","last_active":"2024-04-25T20:58:45.923091+0000","last_peered":"2024-04-25T20:58:45.923091+0000","last_clean":"2024-04-25T20:58:45.923091+0000","last_became_active":"2024-04-25T20:56:51.275087+0000","last_became_peered":"2024-04-25T20:56:51.275087+0000","last_unstale":"2024-04-25T20:58:45.923091+0000","last_undegraded":"2024-04-25T20:58:45.923091+0000","last_fullsized":"2024-04-25T20:58:45.923091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:01:02.514490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524270+0000","last_change":"2024-04-25T20:56:49.031177+0000","last_active":"2024-04-25T20:57:47.524270+0000","last_peered":"2024-04-25T20:57:47.524270+0000","last_clean":"2024-04-25T20:57:47.524270+0000","last_became_active":"2024-04-25T20:56:49.030697+0000","last_became_peered":"2024-04-25T20:56:49.030697+0000","last_unstale":"2024-04-25T20:57:47.524270+0000","last_undegraded":"2024-04-25T20:57:47.524270+0000","last_fullsized":"2024-04-25T20:57:47.524270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:13.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.926293+0000","last_change":"2024-04-25T20:57:47.795434+0000","last_active":"2024-04-25T20:58:45.926293+0000","last_peered":"2024-04-25T20:58:45.926293+0000","last_clean":"2024-04-25T20:58:45.926293+0000","last_became_active":"2024-04-25T20:57:47.795036+0000","last_became_peered":"2024-04-25T20:57:47.795036+0000","last_unstale":"2024-04-25T20:58:45.926293+0000","last_undegraded":"2024-04-25T20:58:45.926293+0000","last_fullsized":"2024-04-25T20:58:45.926293+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:48:56.749328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299385+0000","last_change":"2024-04-25T20:56:48.946689+0000","last_active":"2024-04-25T20:57:48.299385+0000","last_peered":"2024-04-25T20:57:48.299385+0000","last_clean":"2024-04-25T20:57:48.299385+0000","last_became_active":"2024-04-25T20:56:48.946229+0000","last_became_peered":"2024-04-25T20:56:48.946229+0000","last_unstale":"2024-04-25T20:57:48.299385+0000","last_undegraded":"2024-04-25T20:57:48.299385+0000","last_fullsized":"2024-04-25T20:57:48.299385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:04.120197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.883847+0000","last_change":"2024-04-25T20:56:51.275657+0000","last_active":"2024-04-25T20:58:45.883847+0000","last_peered":"2024-04-25T20:58:45.883847+0000","last_clean":"2024-04-25T20:58:45.883847+0000","last_became_active":"2024-04-25T20:56:51.275237+0000","last_became_peered":"2024-04-25T20:56:51.275237+0000","last_unstale":"2024-04-25T20:58:45.883847+0000","last_undegraded":"2024-04-25T20:58:45.883847+0000","last_fullsized":"2024-04-25T20:58:45.883847+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:39.778340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.525606+0000","last_change":"2024-04-25T20:56:48.251882+0000","last_active":"2024-04-25T20:57:47.525606+0000","last_peered":"2024-04-25T20:57:47.525606+0000","last_clean":"2024-04-25T20:57:47.525606+0000","last_became_active":"2024-04-25T20:56:48.251571+0000","last_became_peered":"2024-04-25T20:56:48.251571+0000","last_unstale":"2024-04-25T20:57:47.525606+0000","last_undegraded":"2024-04-25T20:57:47.525606+0000","last_fullsized":"2024-04-25T20:57:47.525606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:51.769332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.926322+0000","last_change":"2024-04-25T20:56:51.280648+0000","last_active":"2024-04-25T20:58:45.926322+0000","last_peered":"2024-04-25T20:58:45.926322+0000","last_clean":"2024-04-25T20:58:45.926322+0000","last_became_active":"2024-04-25T20:56:51.280232+0000","last_became_peered":"2024-04-25T20:56:51.280232+0000","last_unstale":"2024-04-25T20:58:45.926322+0000","last_undegraded":"2024-04-25T20:58:45.926322+0000","last_fullsized":"2024-04-25T20:58:45.926322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:12.375519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.535653+0000","last_change":"2024-04-25T20:57:47.535216+0000","last_active":"2024-04-25T20:57:47.535653+0000","last_peered":"2024-04-25T20:57:47.535653+0000","last_clean":"2024-04-25T20:57:47.535653+0000","last_became_active":"2024-04-25T20:57:47.530348+0000","last_became_peered":"2024-04-25T20:57:47.530348+0000","last_unstale":"2024-04-25T20:57:47.535653+0000","last_undegraded":"2024-04-25T20:57:47.535653+0000","last_fullsized":"2024-04-25T20:57:47.535653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:01.373541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.925775+0000","last_change":"2024-04-25T20:56:51.277166+0000","last_active":"2024-04-25T20:58:45.925775+0000","last_peered":"2024-04-25T20:58:45.925775+0000","last_clean":"2024-04-25T20:58:45.925775+0000","last_became_active":"2024-04-25T20:56:51.276840+0000","last_became_peered":"2024-04-25T20:56:51.276840+0000","last_unstale":"2024-04-25T20:58:45.925775+0000","last_undegraded":"2024-04-25T20:58:45.925775+0000","last_fullsized":"2024-04-25T20:58:45.925775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:52.756660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.526676+0000","last_change":"2024-04-25T20:56:48.263426+0000","last_active":"2024-04-25T20:57:47.526676+0000","last_peered":"2024-04-25T20:57:47.526676+0000","last_clean":"2024-04-25T20:57:47.526676+0000","last_became_active":"2024-04-25T20:56:48.262832+0000","last_became_peered":"2024-04-25T20:56:48.262832+0000","last_unstale":"2024-04-25T20:57:47.526676+0000","last_undegraded":"2024-04-25T20:57:47.526676+0000","last_fullsized":"2024-04-25T20:57:47.526676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:09.362161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.930934+0000","last_change":"2024-04-25T20:56:51.274267+0000","last_active":"2024-04-25T20:58:45.930934+0000","last_peered":"2024-04-25T20:58:45.930934+0000","last_clean":"2024-04-25T20:58:45.930934+0000","last_became_active":"2024-04-25T20:56:51.273807+0000","last_became_peered":"2024-04-25T20:56:51.273807+0000","last_unstale":"2024-04-25T20:58:45.930934+0000","last_undegraded":"2024-04-25T20:58:45.930934+0000","last_fullsized":"2024-04-25T20:58:45.930934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524443+0000","last_change":"2024-04-25T20:56:49.031011+0000","last_active":"2024-04-25T20:57:47.524443+0000","last_peered":"2024-04-25T20:57:47.524443+0000","last_clean":"2024-04-25T20:57:47.524443+0000","last_became_active":"2024-04-25T20:56:49.030522+0000","last_became_peered":"2024-04-25T20:56:49.030522+0000","last_unstale":"2024-04-25T20:57:47.524443+0000","last_undegraded":"2024-04-25T20:57:47.524443+0000","last_fullsized":"2024-04-25T20:57:47.524443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:00.915454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.932778+0000","last_change":"2024-04-25T20:56:51.276989+0000","last_active":"2024-04-25T20:58:45.932778+0000","last_peered":"2024-04-25T20:58:45.932778+0000","last_clean":"2024-04-25T20:58:45.932778+0000","last_became_active":"2024-04-25T20:56:51.275417+0000","last_became_peered":"2024-04-25T20:56:51.275417+0000","last_unstale":"2024-04-25T20:58:45.932778+0000","last_undegraded":"2024-04-25T20:58:45.932778+0000","last_fullsized":"2024-04-25T20:58:45.932778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:27.516062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.591582+0000","last_change":"2024-04-25T20:56:48.253713+0000","last_active":"2024-04-25T20:57:47.591582+0000","last_peered":"2024-04-25T20:57:47.591582+0000","last_clean":"2024-04-25T20:57:47.591582+0000","last_became_active":"2024-04-25T20:56:48.253215+0000","last_became_peered":"2024-04-25T20:56:48.253215+0000","last_unstale":"2024-04-25T20:57:47.591582+0000","last_undegraded":"2024-04-25T20:57:47.591582+0000","last_fullsized":"2024-04-25T20:57:47.591582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:57.946840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.917916+0000","last_change":"2024-04-25T20:56:51.274691+0000","last_active":"2024-04-25T20:58:45.917916+0000","last_peered":"2024-04-25T20:58:45.917916+0000","last_clean":"2024-04-25T20:58:45.917916+0000","last_became_active":"2024-04-25T20:56:51.274350+0000","last_became_peered":"2024-04-25T20:56:51.274350+0000","last_unstale":"2024-04-25T20:58:45.917916+0000","last_undegraded":"2024-04-25T20:58:45.917916+0000","last_fullsized":"2024-04-25T20:58:45.917916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:52.375409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.794071+0000","last_change":"2024-04-25T20:56:48.251616+0000","last_active":"2024-04-25T20:57:47.794071+0000","last_peered":"2024-04-25T20:57:47.794071+0000","last_clean":"2024-04-25T20:57:47.794071+0000","last_became_active":"2024-04-25T20:56:48.251248+0000","last_became_peered":"2024-04-25T20:56:48.251248+0000","last_unstale":"2024-04-25T20:57:47.794071+0000","last_undegraded":"2024-04-25T20:57:47.794071+0000","last_fullsized":"2024-04-25T20:57:47.794071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:51.032356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.919217+0000","last_change":"2024-04-25T20:56:51.278372+0000","last_active":"2024-04-25T20:58:45.919217+0000","last_peered":"2024-04-25T20:58:45.919217+0000","last_clean":"2024-04-25T20:58:45.919217+0000","last_became_active":"2024-04-25T20:56:51.277989+0000","last_became_peered":"2024-04-25T20:56:51.277989+0000","last_unstale":"2024-04-25T20:58:45.919217+0000","last_undegraded":"2024-04-25T20:58:45.919217+0000","last_fullsized":"2024-04-25T20:58:45.919217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.017713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.922892+0000","last_change":"2024-04-25T20:56:51.276270+0000","last_active":"2024-04-25T20:58:45.922892+0000","last_peered":"2024-04-25T20:58:45.922892+0000","last_clean":"2024-04-25T20:58:45.922892+0000","last_became_active":"2024-04-25T20:56:51.275948+0000","last_became_peered":"2024-04-25T20:56:51.275948+0000","last_unstale":"2024-04-25T20:58:45.922892+0000","last_undegraded":"2024-04-25T20:58:45.922892+0000","last_fullsized":"2024-04-25T20:58:45.922892+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:33.548439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524213+0000","last_change":"2024-04-25T20:56:51.283931+0000","last_active":"2024-04-25T20:57:47.524213+0000","last_peered":"2024-04-25T20:57:47.524213+0000","last_clean":"2024-04-25T20:57:47.524213+0000","last_became_active":"2024-04-25T20:56:51.283120+0000","last_became_peered":"2024-04-25T20:56:51.283120+0000","last_unstale":"2024-04-25T20:57:47.524213+0000","last_undegraded":"2024-04-25T20:57:47.524213+0000","last_fullsized":"2024-04-25T20:57:47.524213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:26.676750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524219+0000","last_change":"2024-04-25T20:56:51.286013+0000","last_active":"2024-04-25T20:57:47.524219+0000","last_peered":"2024-04-25T20:57:47.524219+0000","last_clean":"2024-04-25T20:57:47.524219+0000","last_became_active":"2024-04-25T20:56:51.285645+0000","last_became_peered":"2024-04-25T20:56:51.285645+0000","last_unstale":"2024-04-25T20:57:47.524219+0000","last_undegraded":"2024-04-25T20:57:47.524219+0000","last_fullsized":"2024-04-25T20:57:47.524219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:51.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.928815+0000","last_change":"2024-04-25T20:56:51.272567+0000","last_active":"2024-04-25T20:58:45.928815+0000","last_peered":"2024-04-25T20:58:45.928815+0000","last_clean":"2024-04-25T20:58:45.928815+0000","last_became_active":"2024-04-25T20:56:51.271427+0000","last_became_peered":"2024-04-25T20:56:51.271427+0000","last_unstale":"2024-04-25T20:58:45.928815+0000","last_undegraded":"2024-04-25T20:58:45.928815+0000","last_fullsized":"2024-04-25T20:58:45.928815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:19.945982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.793917+0000","last_change":"2024-04-25T20:56:51.272448+0000","last_active":"2024-04-25T20:57:47.793917+0000","last_peered":"2024-04-25T20:57:47.793917+0000","last_clean":"2024-04-25T20:57:47.793917+0000","last_became_active":"2024-04-25T20:56:51.271965+0000","last_became_peered":"2024-04-25T20:56:51.271965+0000","last_unstale":"2024-04-25T20:57:47.793917+0000","last_undegraded":"2024-04-25T20:57:47.793917+0000","last_fullsized":"2024-04-25T20:57:47.793917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:03.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.921914+0000","last_change":"2024-04-25T20:56:51.280260+0000","last_active":"2024-04-25T20:58:45.921914+0000","last_peered":"2024-04-25T20:58:45.921914+0000","last_clean":"2024-04-25T20:58:45.921914+0000","last_became_active":"2024-04-25T20:56:51.279864+0000","last_became_peered":"2024-04-25T20:56:51.279864+0000","last_unstale":"2024-04-25T20:58:45.921914+0000","last_undegraded":"2024-04-25T20:58:45.921914+0000","last_fullsized":"2024-04-25T20:58:45.921914+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.136824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:47.524461+0000","last_change":"2024-04-25T20:56:51.279041+0000","last_active":"2024-04-25T20:57:47.524461+0000","last_peered":"2024-04-25T20:57:47.524461+0000","last_clean":"2024-04-25T20:57:47.524461+0000","last_became_active":"2024-04-25T20:56:51.278302+0000","last_became_peered":"2024-04-25T20:56:51.278302+0000","last_unstale":"2024-04-25T20:57:47.524461+0000","last_undegraded":"2024-04-25T20:57:47.524461+0000","last_fullsized":"2024-04-25T20:57:47.524461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:02.510919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.913104+0000","last_change":"2024-04-25T20:56:51.275618+0000","last_active":"2024-04-25T20:58:45.913104+0000","last_peered":"2024-04-25T20:58:45.913104+0000","last_clean":"2024-04-25T20:58:45.913104+0000","last_became_active":"2024-04-25T20:56:51.275249+0000","last_became_peered":"2024-04-25T20:56:51.275249+0000","last_unstale":"2024-04-25T20:58:45.913104+0000","last_undegraded":"2024-04-25T20:58:45.913104+0000","last_fullsized":"2024-04-25T20:58:45.913104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:01.793083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.912867+0000","last_change":"2024-04-25T20:56:51.275868+0000","last_active":"2024-04-25T20:58:45.912867+0000","last_peered":"2024-04-25T20:58:45.912867+0000","last_clean":"2024-04-25T20:58:45.912867+0000","last_became_active":"2024-04-25T20:56:51.275544+0000","last_became_peered":"2024-04-25T20:56:51.275544+0000","last_unstale":"2024-04-25T20:58:45.912867+0000","last_undegraded":"2024-04-25T20:58:45.912867+0000","last_fullsized":"2024-04-25T20:58:45.912867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:31.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.923557+0000","last_change":"2024-04-25T20:56:51.274963+0000","last_active":"2024-04-25T20:58:45.923557+0000","last_peered":"2024-04-25T20:58:45.923557+0000","last_clean":"2024-04-25T20:58:45.923557+0000","last_became_active":"2024-04-25T20:56:51.274597+0000","last_became_peered":"2024-04-25T20:56:51.274597+0000","last_unstale":"2024-04-25T20:58:45.923557+0000","last_undegraded":"2024-04-25T20:58:45.923557+0000","last_fullsized":"2024-04-25T20:58:45.923557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:14.464560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:57:48.299653+0000","last_change":"2024-04-25T20:56:51.279602+0000","last_active":"2024-04-25T20:57:48.299653+0000","last_peered":"2024-04-25T20:57:48.299653+0000","last_clean":"2024-04-25T20:57:48.299653+0000","last_became_active":"2024-04-25T20:56:51.279221+0000","last_became_peered":"2024-04-25T20:56:51.279221+0000","last_unstale":"2024-04-25T20:57:48.299653+0000","last_undegraded":"2024-04-25T20:57:48.299653+0000","last_fullsized":"2024-04-25T20:57:48.299653+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:26.960492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.910363+0000","last_change":"2024-04-25T20:56:51.280465+0000","last_active":"2024-04-25T20:58:45.910363+0000","last_peered":"2024-04-25T20:58:45.910363+0000","last_clean":"2024-04-25T20:58:45.910363+0000","last_became_active":"2024-04-25T20:56:51.280168+0000","last_became_peered":"2024-04-25T20:56:51.280168+0000","last_unstale":"2024-04-25T20:58:45.910363+0000","last_undegraded":"2024-04-25T20:58:45.910363+0000","last_fullsized":"2024-04-25T20:58:45.910363+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.211113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:58:45.923686+0000","last_change":"2024-04-25T20:56:51.283084+0000","last_active":"2024-04-25T20:58:45.923686+0000","last_peered":"2024-04-25T20:58:45.923686+0000","last_clean":"2024-04-25T20:58:45.923686+0000","last_became_active":"2024-04-25T20:56:51.282692+0000","last_became_peered":"2024-04-25T20:56:51.282692+0000","last_unstale":"2024-04-25T20:58:45.923686+0000","last_undegraded":"2024-04-25T20:58:45.923686+0000","last_fullsized":"2024-04-25T20:58:45.923686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:13:27.527112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":363,"ondisk_log_size":363,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65021,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":620,"num_read_kb":377,"num_write":170,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":484239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579949,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1695744,"data_stored":1136458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":484475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":484395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1076998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40484,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704668,"statfs":{"total":95995035648,"available":95953580032,"internally_reserved":0,"allocated":1003520,"data_stored":483864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":544004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":485135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":545745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":484175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":484833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32760,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712392,"statfs":{"total":95995035648,"available":95961489408,"internally_reserved":0,"allocated":1622016,"data_stored":1076150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:58:51.317 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T20:58:51.317 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T20:58:51.317 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T20:58:51.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T20:58:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:51.943 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:51 smithi047 ceph-mon[29195]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T20:58:51.944 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:51 smithi047 ceph-mon[29195]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:52.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:51 smithi092 ceph-mon[32524]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T20:58:52.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:51 smithi092 ceph-mon[32524]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:52.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:51 smithi186 ceph-mon[32529]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T20:58:52.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:51 smithi186 ceph-mon[32529]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:53.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:52 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:53.018 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:52 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:53.029 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:52 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 -- 172.21.15.47:0/3700687394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8102300 msgr2=0x7f5be810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3700687394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8102300 0x7f5be810e7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5be000b0a0 tx=0x7f5be000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 -- 172.21.15.47:0/3700687394 shutdown_connections 2024-04-25T20:58:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3700687394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8102300 0x7f5be810e7f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3700687394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5be8107990 0x7f5be8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3700687394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5be8101940 0x7f5be8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 -- 172.21.15.47:0/3700687394 >> 172.21.15.47:0/3700687394 conn(0x7f5be80788f0 msgr2=0x7f5be8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 -- 172.21.15.47:0/3700687394 shutdown_connections 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.202+0000 7f5befa2d700 1 -- 172.21.15.47:0/3700687394 wait complete. 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5befa2d700 1 Processor -- start 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5befa2d700 1 -- start start 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5befa2d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8101940 0x7f5be8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5befa2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5be8102300 0x7f5be81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5befa2d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5be8107990 0x7f5be819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5befa2d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8111b00 con 0x7f5be8101940 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5befa2d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8111980 con 0x7f5be8102300 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5befa2d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8111c80 con 0x7f5be8107990 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5bed7c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8101940 0x7f5be8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5bed7c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8101940 0x7f5be8199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48050/0 (socket says 172.21.15.47:48050) 2024-04-25T20:58:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.203+0000 7f5bed7c9700 1 -- 172.21.15.47:0/3048679257 learned_addr learned my addr 172.21.15.47:0/3048679257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bedfca700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5be8107990 0x7f5be819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5becfc8700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5be8102300 0x7f5be81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bed7c9700 1 -- 172.21.15.47:0/3048679257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5be8107990 msgr2=0x7f5be819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bed7c9700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5be8107990 0x7f5be819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bed7c9700 1 -- 172.21.15.47:0/3048679257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5be8102300 msgr2=0x7f5be81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bed7c9700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5be8102300 0x7f5be81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bed7c9700 1 -- 172.21.15.47:0/3048679257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be819e090 con 0x7f5be8101940 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bed7c9700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8101940 0x7f5be8199080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5be400d980 tx=0x7f5be400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bde7fc700 1 -- 172.21.15.47:0/3048679257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4017070 con 0x7f5be8101940 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bde7fc700 1 -- 172.21.15.47:0/3048679257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be4005020 con 0x7f5be8101940 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be819e380 con 0x7f5be8101940 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5bde7fc700 1 -- 172.21.15.47:0/3048679257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4009b50 con 0x7f5be8101940 2024-04-25T20:58:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.204+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be81a5c00 con 0x7f5be8101940 2024-04-25T20:58:53.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.205+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be8069730 con 0x7f5be8101940 2024-04-25T20:58:53.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.206+0000 7f5bde7fc700 1 -- 172.21.15.47:0/3048679257 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5be40051d0 con 0x7f5be8101940 2024-04-25T20:58:53.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.207+0000 7f5bde7fc700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:53.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.207+0000 7f5becfc8700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:53.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.207+0000 7f5bde7fc700 1 -- 172.21.15.47:0/3048679257 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f5be40d0650 con 0x7f5be8101940 2024-04-25T20:58:53.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.208+0000 7f5becfc8700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8005950 tx=0x7f5bd80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.211+0000 7f5bde7fc700 1 -- 172.21.15.47:0/3048679257 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5be4020020 con 0x7f5be8101940 2024-04-25T20:58:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.476+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5be802df80 con 0x7f5be8101940 2024-04-25T20:58:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.477+0000 7f5bde7fc700 1 -- 172.21.15.47:0/3048679257 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5be409d040 con 0x7f5be8101940 2024-04-25T20:58:53.478 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:58:53.478 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T20:58:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5bd40a3cc0 msgr2=0x7f5bd40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8005950 tx=0x7f5bd80058e0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8101940 msgr2=0x7f5be8199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8101940 0x7f5be8199080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5be400d980 tx=0x7f5be400dc90 comp rx=0 tx=0).stop 2024-04-25T20:58:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 shutdown_connections 2024-04-25T20:58:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be8101940 0x7f5be8199080 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5be8107990 0x7f5be819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 --2- 172.21.15.47:0/3048679257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5be8102300 0x7f5be81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 >> 172.21.15.47:0/3048679257 conn(0x7f5be80788f0 msgr2=0x7f5be806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 shutdown_connections 2024-04-25T20:58:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:53.479+0000 7f5befa2d700 1 -- 172.21.15.47:0/3048679257 wait complete. 2024-04-25T20:58:54.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:53 smithi092 ceph-mon[32524]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 596 B/s wr, 7 op/s 2024-04-25T20:58:54.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:53 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3048679257' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T20:58:54.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:53 smithi047 ceph-mon[29195]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 596 B/s wr, 7 op/s 2024-04-25T20:58:54.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:53 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3048679257' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T20:58:54.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:53 smithi186 ceph-mon[32529]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 596 B/s wr, 7 op/s 2024-04-25T20:58:54.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:53 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3048679257' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T20:58:54.533 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T20:58:54.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:58:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:55.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:54 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:54 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:54 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:54 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:54 smithi047 ceph-mon[29195]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 596 B/s wr, 6 op/s 2024-04-25T20:58:55.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:54 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:54 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:54 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:54 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:54 smithi186 ceph-mon[32529]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 596 B/s wr, 6 op/s 2024-04-25T20:58:55.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:54 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:54 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:54 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:54 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:55.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:54 smithi092 ceph-mon[32524]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 596 B/s wr, 6 op/s 2024-04-25T20:58:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 -- 172.21.15.47:0/1490853433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8103450 msgr2=0x7f51f810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/1490853433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8103450 0x7f51f810f8e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f51f000b0a0 tx=0x7f51f000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 -- 172.21.15.47:0/1490853433 shutdown_connections 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/1490853433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8103450 0x7f51f810f8e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/1490853433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f8108a80 0x7f51f8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/1490853433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51f8102a90 0x7f51f8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 -- 172.21.15.47:0/1490853433 >> 172.21.15.47:0/1490853433 conn(0x7f51f8079bf0 msgr2=0x7f51f807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 -- 172.21.15.47:0/1490853433 shutdown_connections 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.481+0000 7f51fc9ec700 1 -- 172.21.15.47:0/1490853433 wait complete. 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51fc9ec700 1 Processor -- start 2024-04-25T20:58:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51fc9ec700 1 -- start start 2024-04-25T20:58:56.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51fc9ec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8102a90 0x7f51f8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:56.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51fc9ec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51f8103450 0x7f51f8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:56.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51fc9ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f8108a80 0x7f51f80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:56.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51fc9ec700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f51f8112c90 con 0x7f51f8102a90 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51fc9ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f51f8112b10 con 0x7f51f8108a80 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51fc9ec700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f51f8112e10 con 0x7f51f8103450 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51f659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8102a90 0x7f51f8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51f659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8102a90 0x7f51f8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49760/0 (socket says 172.21.15.47:49760) 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.482+0000 7f51f659c700 1 -- 172.21.15.47:0/163908284 learned_addr learned my addr 172.21.15.47:0/163908284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f5d9b700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51f8103450 0x7f51f8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f6d9d700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f8108a80 0x7f51f80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f659c700 1 -- 172.21.15.47:0/163908284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51f8103450 msgr2=0x7f51f8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f659c700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51f8103450 0x7f51f8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f659c700 1 -- 172.21.15.47:0/163908284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f8108a80 msgr2=0x7f51f80757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f659c700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f8108a80 0x7f51f80757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f659c700 1 -- 172.21.15.47:0/163908284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f81a7660 con 0x7f51f8102a90 2024-04-25T20:58:56.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f6d9d700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f8108a80 0x7f51f80757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51f659c700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8102a90 0x7f51f8074d40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f51e800b530 tx=0x7f51e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51e77fe700 1 -- 172.21.15.47:0/163908284 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8015020 con 0x7f51f8102a90 2024-04-25T20:58:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51e77fe700 1 -- 172.21.15.47:0/163908284 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e8011ce0 con 0x7f51f8102a90 2024-04-25T20:58:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f81a7950 con 0x7f51f8102a90 2024-04-25T20:58:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51e77fe700 1 -- 172.21.15.47:0/163908284 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8010e40 con 0x7f51f8102a90 2024-04-25T20:58:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.483+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f81a7e90 con 0x7f51f8102a90 2024-04-25T20:58:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.484+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f8069730 con 0x7f51f8102a90 2024-04-25T20:58:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.485+0000 7f51e77fe700 1 -- 172.21.15.47:0/163908284 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f51e8018020 con 0x7f51f8102a90 2024-04-25T20:58:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.486+0000 7f51e77fe700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f51e00a3c40 0x7f51e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.486+0000 7f51f5d9b700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f51e00a3c40 0x7f51e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.486+0000 7f51e77fe700 1 -- 172.21.15.47:0/163908284 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f51e80cfc30 con 0x7f51f8102a90 2024-04-25T20:58:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.486+0000 7f51f5d9b700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f51e00a3c40 0x7f51e00a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51f807a260 tx=0x7f51ec0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.490+0000 7f51e77fe700 1 -- 172.21.15.47:0/163908284 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51e801f020 con 0x7f51f8102a90 2024-04-25T20:58:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:56 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:56 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:56 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:56.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:56 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:56.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:56 smithi186 ceph-mon[32529]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.744+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51f802df80 con 0x7f51f8102a90 2024-04-25T20:58:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.745+0000 7f51e77fe700 1 -- 172.21.15.47:0/163908284 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11853 (secure 0 0 0) 0x7f51e809c600 con 0x7f51f8102a90 2024-04-25T20:58:56.746 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:58:56.746 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:57:11.813557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24488,24467]},"id":1}]} 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f51e00a3c40 msgr2=0x7f51e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f51e00a3c40 0x7f51e00a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51f807a260 tx=0x7f51ec0073d0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8102a90 msgr2=0x7f51f8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8102a90 0x7f51f8074d40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f51e800b530 tx=0x7f51e800b840 comp rx=0 tx=0).stop 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 shutdown_connections 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f51e00a3c40 0x7f51e00a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f8102a90 0x7f51f8074d40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f8108a80 0x7f51f80757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 --2- 172.21.15.47:0/163908284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51f8103450 0x7f51f8075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 >> 172.21.15.47:0/163908284 conn(0x7f51f8079bf0 msgr2=0x7f51f81016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 shutdown_connections 2024-04-25T20:58:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:56.747+0000 7f51fc9ec700 1 -- 172.21.15.47:0/163908284 wait complete. 2024-04-25T20:58:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2024-04-25T20:58:56.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:56 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:56.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:56 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:56.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:56 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:56.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:56 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:56.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:56 smithi047 ceph-mon[29195]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:57.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:56 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:57.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:56 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:57.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:56 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:57.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:56 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:57.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:56 smithi092 ceph-mon[32524]: from='mgr.14152 172.21.15.47:0/3412491784' entity='mgr.x' 2024-04-25T20:58:57.438 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:58:57.700 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:57.728 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:57 smithi047 ceph-mon[29195]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 596 B/s wr, 7 op/s 2024-04-25T20:58:57.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:57 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/163908284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:57 smithi186 ceph-mon[32529]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 596 B/s wr, 7 op/s 2024-04-25T20:58:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:57 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/163908284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:58.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:57 smithi092 ceph-mon[32524]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 596 B/s wr, 7 op/s 2024-04-25T20:58:58.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:57 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/163908284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 -- 172.21.15.47:0/906285507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa098108a20 msgr2=0x7fa098108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 --2- 172.21.15.47:0/906285507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa098108a20 0x7fa098108e00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa088009990 tx=0x7fa088009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 -- 172.21.15.47:0/906285507 shutdown_connections 2024-04-25T20:58:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 --2- 172.21.15.47:0/906285507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa098108a20 0x7fa098108e00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 --2- 172.21.15.47:0/906285507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa09806b980 0x7fa09810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 --2- 172.21.15.47:0/906285507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa09806afe0 0x7fa09806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 -- 172.21.15.47:0/906285507 >> 172.21.15.47:0/906285507 conn(0x7fa0980788b0 msgr2=0x7fa098078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 -- 172.21.15.47:0/906285507 shutdown_connections 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.288+0000 7fa09d491700 1 -- 172.21.15.47:0/906285507 wait complete. 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa09d491700 1 Processor -- start 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa09d491700 1 -- start start 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa09d491700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa09806afe0 0x7fa098101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa09d491700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09806b980 0x7fa098101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa09d491700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa098108a20 0x7fa098102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa09d491700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa098112bd0 con 0x7fa09806b980 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa09d491700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa098112a50 con 0x7fa098108a20 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa09d491700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa098112d50 con 0x7fa09806afe0 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa0967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09806b980 0x7fa098101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa0967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09806b980 0x7fa098101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49776/0 (socket says 172.21.15.47:49776) 2024-04-25T20:58:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.289+0000 7fa0967fc700 1 -- 172.21.15.47:0/2589361693 learned_addr learned my addr 172.21.15.47:0/2589361693 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa096ffd700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa09806afe0 0x7fa098101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa0977fe700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa098108a20 0x7fa098102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa0967fc700 1 -- 172.21.15.47:0/2589361693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa09806afe0 msgr2=0x7fa098101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa0967fc700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa09806afe0 0x7fa098101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa0967fc700 1 -- 172.21.15.47:0/2589361693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa098108a20 msgr2=0x7fa098102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa0967fc700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa098108a20 0x7fa098102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa0967fc700 1 -- 172.21.15.47:0/2589361693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0981a7670 con 0x7fa09806b980 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa096ffd700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa09806afe0 0x7fa098101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa0977fe700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa098108a20 0x7fa098102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa0967fc700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09806b980 0x7fa098101f40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00e9c0 tx=0x7fa08c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa07ffff700 1 -- 172.21.15.47:0/2589361693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00cbd0 con 0x7fa09806b980 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0981a7960 con 0x7fa09806b980 2024-04-25T20:58:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa07ffff700 1 -- 172.21.15.47:0/2589361693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08c00cd70 con 0x7fa09806b980 2024-04-25T20:58:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa07ffff700 1 -- 172.21.15.47:0/2589361693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c005610 con 0x7fa09806b980 2024-04-25T20:58:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.290+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0981a7ea0 con 0x7fa09806b980 2024-04-25T20:58:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.291+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa098069730 con 0x7fa09806b980 2024-04-25T20:58:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.292+0000 7fa07ffff700 1 -- 172.21.15.47:0/2589361693 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa08c004750 con 0x7fa09806b980 2024-04-25T20:58:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.293+0000 7fa07ffff700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa0800a3c40 0x7fa0800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.293+0000 7fa096ffd700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa0800a3c40 0x7fa0800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.293+0000 7fa07ffff700 1 -- 172.21.15.47:0/2589361693 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fa08c0d0740 con 0x7fa09806b980 2024-04-25T20:58:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.293+0000 7fa096ffd700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa0800a3c40 0x7fa0800a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa08800b2e0 tx=0x7fa08801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:59.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.298+0000 7fa07ffff700 1 -- 172.21.15.47:0/2589361693 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa08c09d110 con 0x7fa09806b980 2024-04-25T20:58:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.553+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa09802df80 con 0x7fa09806b980 2024-04-25T20:58:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.554+0000 7fa07ffff700 1 -- 172.21.15.47:0/2589361693 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11853 (secure 0 0 0) 0x7fa08c09ca10 con 0x7fa09806b980 2024-04-25T20:58:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:58:59.555 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:57:11.813557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24488,24467]},"id":1}]} 2024-04-25T20:58:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa0800a3c40 msgr2=0x7fa0800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa0800a3c40 0x7fa0800a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa08800b2e0 tx=0x7fa08801a040 comp rx=0 tx=0).stop 2024-04-25T20:58:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09806b980 msgr2=0x7fa098101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09806b980 0x7fa098101f40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00e9c0 tx=0x7fa08c00ecd0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 shutdown_connections 2024-04-25T20:58:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa0800a3c40 0x7fa0800a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa098108a20 0x7fa098102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa09806afe0 0x7fa098101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 --2- 172.21.15.47:0/2589361693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa09806b980 0x7fa098101f40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.556+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 >> 172.21.15.47:0/2589361693 conn(0x7fa0980788b0 msgr2=0x7fa0980ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.557+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 shutdown_connections 2024-04-25T20:58:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:58:59.557+0000 7fa09d491700 1 -- 172.21.15.47:0/2589361693 wait complete. 2024-04-25T20:58:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2024-04-25T20:58:59.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:59 smithi047 ceph-mon[29195]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:58:59.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:58:59 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2589361693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:59.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:59 smithi186 ceph-mon[32529]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:58:59.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:58:59 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2589361693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:00.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:59 smithi092 ceph-mon[32524]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:59:00.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:58:59 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2589361693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:00.258 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:59:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:01.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:01 smithi047 ceph-mon[29195]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T20:59:01.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:01 smithi186 ceph-mon[32529]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T20:59:02.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:01 smithi092 ceph-mon[32524]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 -- 172.21.15.47:0/2459049383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704102ac0 msgr2=0x7f1704102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 --2- 172.21.15.47:0/2459049383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704102ac0 0x7f1704102f40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009990 tx=0x7f16f8009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 -- 172.21.15.47:0/2459049383 shutdown_connections 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 --2- 172.21.15.47:0/2459049383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1704103480 0x7f170410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 --2- 172.21.15.47:0/2459049383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1704108a70 0x7f1704108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 --2- 172.21.15.47:0/2459049383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704102ac0 0x7f1704102f40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 -- 172.21.15.47:0/2459049383 >> 172.21.15.47:0/2459049383 conn(0x7f1704079bb0 msgr2=0x7f1704079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 -- 172.21.15.47:0/2459049383 shutdown_connections 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.275+0000 7f1709741700 1 -- 172.21.15.47:0/2459049383 wait complete. 2024-04-25T20:59:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.276+0000 7f1709741700 1 Processor -- start 2024-04-25T20:59:02.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.276+0000 7f1709741700 1 -- start start 2024-04-25T20:59:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.276+0000 7f1709741700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1704102ac0 0x7f1704199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.276+0000 7f1709741700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704103480 0x7f1704199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.276+0000 7f1709741700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1704108a70 0x7f170419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.276+0000 7f1709741700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1704112c00 con 0x7f1704103480 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f1709741700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1704112a80 con 0x7f1704108a70 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f1709741700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1704112d80 con 0x7f1704102ac0 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704103480 0x7f1704199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704103480 0x7f1704199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49798/0 (socket says 172.21.15.47:49798) 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 -- 172.21.15.47:0/2294364449 learned_addr learned my addr 172.21.15.47:0/2294364449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f1702ffd700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1704102ac0 0x7f1704199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17037fe700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1704108a70 0x7f170419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 -- 172.21.15.47:0/2294364449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1704102ac0 msgr2=0x7f1704199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1704102ac0 0x7f1704199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 -- 172.21.15.47:0/2294364449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1704108a70 msgr2=0x7f170419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1704108a70 0x7f170419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 -- 172.21.15.47:0/2294364449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170419e060 con 0x7f1704103480 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17037fe700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1704108a70 0x7f170419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f17027fc700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704103480 0x7f1704199590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f16f8000c00 tx=0x7f16f80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f16ebfff700 1 -- 172.21.15.47:0/2294364449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f8027070 con 0x7f1704103480 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170419e2f0 con 0x7f1704103480 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f16ebfff700 1 -- 172.21.15.47:0/2294364449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f80040e0 con 0x7f1704103480 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f16ebfff700 1 -- 172.21.15.47:0/2294364449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f800f460 con 0x7f1704103480 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.277+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041a5bd0 con 0x7f1704103480 2024-04-25T20:59:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.278+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1704069730 con 0x7f1704103480 2024-04-25T20:59:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.280+0000 7f16ebfff700 1 -- 172.21.15.47:0/2294364449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f16f802a070 con 0x7f1704103480 2024-04-25T20:59:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.280+0000 7f16ebfff700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f16ec0a3c40 0x7f16ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.280+0000 7f1702ffd700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f16ec0a3c40 0x7f16ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.281+0000 7f16ebfff700 1 -- 172.21.15.47:0/2294364449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f16f80d1820 con 0x7f1704103480 2024-04-25T20:59:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.281+0000 7f1702ffd700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f16ec0a3c40 0x7f16ec0a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005950 tx=0x7f16f400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:02.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.284+0000 7f16ebfff700 1 -- 172.21.15.47:0/2294364449 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16f8031070 con 0x7f1704103480 2024-04-25T20:59:02.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.494+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f170402df80 con 0x7f1704103480 2024-04-25T20:59:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.496+0000 7f16ebfff700 1 -- 172.21.15.47:0/2294364449 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22599 (secure 0 0 0) 0x7f16f8013070 con 0x7f1704103480 2024-04-25T20:59:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:02.497 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:57:47.520070+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:59:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.498+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f16ec0a3c40 msgr2=0x7f16ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.498+0000 7f1709741700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f16ec0a3c40 0x7f16ec0a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005950 tx=0x7f16f400a300 comp rx=0 tx=0).stop 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.498+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704103480 msgr2=0x7f1704199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.498+0000 7f1709741700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704103480 0x7f1704199590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f16f8000c00 tx=0x7f16f80047b0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.499+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 shutdown_connections 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.499+0000 7f1709741700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f16ec0a3c40 0x7f16ec0a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.499+0000 7f1709741700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1704108a70 0x7f170419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.499+0000 7f1709741700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1704102ac0 0x7f1704199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.499+0000 7f1709741700 1 --2- 172.21.15.47:0/2294364449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1704103480 0x7f1704199590 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.499+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 >> 172.21.15.47:0/2294364449 conn(0x7f1704079bb0 msgr2=0x7f1704101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.499+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 shutdown_connections 2024-04-25T20:59:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:02.499+0000 7f1709741700 1 -- 172.21.15.47:0/2294364449 wait complete. 2024-04-25T20:59:02.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:02 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2294364449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:59:02.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:02 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2294364449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:59:03.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:02 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2294364449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:59:03.195 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:59:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:03.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:03 smithi047 ceph-mon[29195]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-25T20:59:03.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:03 smithi186 ceph-mon[32529]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-25T20:59:04.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:03 smithi092 ceph-mon[32524]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.100+0000 7f54abdbd700 1 -- 172.21.15.47:0/3929152905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4108a30 msgr2=0x7f54a4108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 --2- 172.21.15.47:0/3929152905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4108a30 0x7f54a4108e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009990 tx=0x7f54a0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 -- 172.21.15.47:0/3929152905 shutdown_connections 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 --2- 172.21.15.47:0/3929152905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4108a30 0x7f54a4108e10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 --2- 172.21.15.47:0/3929152905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a406b0f0 0x7f54a410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 --2- 172.21.15.47:0/3929152905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54a406a750 0x7f54a406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 -- 172.21.15.47:0/3929152905 >> 172.21.15.47:0/3929152905 conn(0x7f54a40788f0 msgr2=0x7f54a4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 -- 172.21.15.47:0/3929152905 shutdown_connections 2024-04-25T20:59:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 -- 172.21.15.47:0/3929152905 wait complete. 2024-04-25T20:59:05.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 Processor -- start 2024-04-25T20:59:05.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.101+0000 7f54abdbd700 1 -- start start 2024-04-25T20:59:05.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54abdbd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54a406a750 0x7f54a4101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:05.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54abdbd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a406b0f0 0x7f54a4101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:05.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54abdbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a4108a30 0x7f54a4102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:05.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54abdbd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4112c30 con 0x7f54a406b0f0 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54abdbd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4112ab0 con 0x7f54a4108a30 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54abdbd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4112db0 con 0x7f54a406a750 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a406b0f0 0x7f54a4101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a406b0f0 0x7f54a4101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48218/0 (socket says 172.21.15.47:48218) 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 -- 172.21.15.47:0/863279727 learned_addr learned my addr 172.21.15.47:0/863279727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9b59700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54a406a750 0x7f54a4101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54aa35a700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a4108a30 0x7f54a4102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 -- 172.21.15.47:0/863279727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54a406a750 msgr2=0x7f54a4101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54a406a750 0x7f54a4101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 -- 172.21.15.47:0/863279727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a4108a30 msgr2=0x7f54a4102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a4108a30 0x7f54a4102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 -- 172.21.15.47:0/863279727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a41a7690 con 0x7f54a406b0f0 2024-04-25T20:59:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54a9358700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a406b0f0 0x7f54a4101f50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f549400d950 tx=0x7f549400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f549affd700 1 -- 172.21.15.47:0/863279727 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494016070 con 0x7f54a406b0f0 2024-04-25T20:59:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f549affd700 1 -- 172.21.15.47:0/863279727 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549400fd40 con 0x7f54a406b0f0 2024-04-25T20:59:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.102+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a41a7980 con 0x7f54a406b0f0 2024-04-25T20:59:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.103+0000 7f549affd700 1 -- 172.21.15.47:0/863279727 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549400e3b0 con 0x7f54a406b0f0 2024-04-25T20:59:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.103+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a41a7ec0 con 0x7f54a406b0f0 2024-04-25T20:59:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.103+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a4069730 con 0x7f54a406b0f0 2024-04-25T20:59:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.105+0000 7f549affd700 1 -- 172.21.15.47:0/863279727 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5494019070 con 0x7f54a406b0f0 2024-04-25T20:59:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.105+0000 7f549affd700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f54900a3c40 0x7f54900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.105+0000 7f54a9b59700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f54900a3c40 0x7f54900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.105+0000 7f549affd700 1 -- 172.21.15.47:0/863279727 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f54940cfe80 con 0x7f54a406b0f0 2024-04-25T20:59:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.106+0000 7f54a9b59700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f54900a3c40 0x7f54900a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f54a000b5c0 tx=0x7f54a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:05.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.109+0000 7f549affd700 1 -- 172.21.15.47:0/863279727 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f549409c850 con 0x7f54a406b0f0 2024-04-25T20:59:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.365+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54a402df80 con 0x7f54a406b0f0 2024-04-25T20:59:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.366+0000 7f549affd700 1 -- 172.21.15.47:0/863279727 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11853 (secure 0 0 0) 0x7f549409c150 con 0x7f54a406b0f0 2024-04-25T20:59:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:05.367 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:57:11.813557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24488,24467]},"id":1}]} 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f54900a3c40 msgr2=0x7f54900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f54900a3c40 0x7f54900a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f54a000b5c0 tx=0x7f54a0005960 comp rx=0 tx=0).stop 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a406b0f0 msgr2=0x7f54a4101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a406b0f0 0x7f54a4101f50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f549400d950 tx=0x7f549400dc60 comp rx=0 tx=0).stop 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 shutdown_connections 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f54900a3c40 0x7f54900a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a4108a30 0x7f54a4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54a406a750 0x7f54a4101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 --2- 172.21.15.47:0/863279727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a406b0f0 0x7f54a4101f50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 >> 172.21.15.47:0/863279727 conn(0x7f54a40788f0 msgr2=0x7f54a40ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 shutdown_connections 2024-04-25T20:59:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:05.368+0000 7f54abdbd700 1 -- 172.21.15.47:0/863279727 wait complete. 2024-04-25T20:59:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2024-04-25T20:59:05.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:05 smithi047 ceph-mon[29195]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:05.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:05 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/863279727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:05 smithi186 ceph-mon[32529]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:05 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/863279727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:05.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T20:56:53.085968+0000', 'modified': '2024-04-25T20:57:11.813557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24467, 'mds_2': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/3860377657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 3860377657}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 3860377657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/4095064414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 4095064414}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 4095064414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6839/60292465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 60292465}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 60292465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.92:6835/2097108178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 2097108178}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 2097108178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/1984144918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1984144918}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1984144918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/941998037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 941998037}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 941998037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24452, 24488, 24467]} 2024-04-25T20:59:05.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T20:59:05.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.e status 2024-04-25T20:59:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:05 smithi092 ceph-mon[32524]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:05 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/863279727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:06.239 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:07.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:07 smithi047 ceph-mon[29195]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 -- 172.21.15.47:0/1336436829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f309806b1f0 msgr2=0x7f309806b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 --2- 172.21.15.47:0/1336436829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f309806b1f0 0x7f309806b670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f308c009990 tx=0x7f308c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 -- 172.21.15.47:0/1336436829 shutdown_connections 2024-04-25T20:59:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 --2- 172.21.15.47:0/1336436829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f309806bbb0 0x7f309810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 --2- 172.21.15.47:0/1336436829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3098108a60 0x7f3098108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 --2- 172.21.15.47:0/1336436829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f309806b1f0 0x7f309806b670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 -- 172.21.15.47:0/1336436829 >> 172.21.15.47:0/1336436829 conn(0x7f30980789e0 msgr2=0x7f3098078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 -- 172.21.15.47:0/1336436829 shutdown_connections 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.928+0000 7f309d2eb700 1 -- 172.21.15.47:0/1336436829 wait complete. 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.929+0000 7f309d2eb700 1 Processor -- start 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.929+0000 7f309d2eb700 1 -- start start 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.929+0000 7f309d2eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309806b1f0 0x7f30981019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.929+0000 7f309d2eb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f309806bbb0 0x7f3098101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.929+0000 7f309d2eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3098108a60 0x7f3098102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.929+0000 7f309d2eb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3098112c40 con 0x7f3098108a60 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.929+0000 7f309d2eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3098112ac0 con 0x7f309806b1f0 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.929+0000 7f309d2eb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3098112dc0 con 0x7f309806bbb0 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3098108a60 0x7f3098102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3098108a60 0x7f3098102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48242/0 (socket says 172.21.15.47:48242) 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 -- 172.21.15.47:0/2702915012 learned_addr learned my addr 172.21.15.47:0/2702915012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f3096ffd700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309806b1f0 0x7f30981019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30967fc700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f309806bbb0 0x7f3098101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 -- 172.21.15.47:0/2702915012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f309806bbb0 msgr2=0x7f3098101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f309806bbb0 0x7f3098101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 -- 172.21.15.47:0/2702915012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309806b1f0 msgr2=0x7f30981019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309806b1f0 0x7f30981019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 -- 172.21.15.47:0/2702915012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30981a7690 con 0x7f3098108a60 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f30977fe700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3098108a60 0x7f3098102430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f309000ef30 tx=0x7f309000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f307ffff700 1 -- 172.21.15.47:0/2702915012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30900092e0 con 0x7f3098108a60 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f309d2eb700 1 -- 172.21.15.47:0/2702915012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30981a7920 con 0x7f3098108a60 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f307ffff700 1 -- 172.21.15.47:0/2702915012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3090004bb0 con 0x7f3098108a60 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f307ffff700 1 -- 172.21.15.47:0/2702915012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090004020 con 0x7f3098108a60 2024-04-25T20:59:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.930+0000 7f309d2eb700 1 -- 172.21.15.47:0/2702915012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30981a7e80 con 0x7f3098108a60 2024-04-25T20:59:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.932+0000 7f307ffff700 1 -- 172.21.15.47:0/2702915012 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f309001b030 con 0x7f3098108a60 2024-04-25T20:59:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.933+0000 7f307ffff700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f30800a3cc0 0x7f30800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.933+0000 7f307ffff700 1 -- 172.21.15.47:0/2702915012 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f30900cfdd0 con 0x7f3098108a60 2024-04-25T20:59:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.933+0000 7f3096ffd700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f30800a3cc0 0x7f30800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f3096ffd700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f30800a3cc0 0x7f30800a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3088007b60 tx=0x7f30880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f30a7b74b80 1 Processor -- start 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f30a7b74b80 1 -- start start 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 0x564f794dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564f79429ce0 con 0x564f794d77d0 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564f793d0910 con 0x564f794dc580 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564f79425ca0 con 0x564f794d80f0 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794d7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48254/0 (socket says 172.21.15.47:48254) 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.934+0000 7f3096ffd700 1 -- 172.21.15.47:0/1475692736 learned_addr learned my addr 172.21.15.47:0/1475692736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f30967fc700 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 0x564f794dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f3096ffd700 1 -- 172.21.15.47:0/1475692736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 msgr2=0x564f794dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f3096ffd700 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 0x564f794dc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f3096ffd700 1 -- 172.21.15.47:0/1475692736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 msgr2=0x564f794dc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f30977fe700 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f3096ffd700 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f3096ffd700 1 -- 172.21.15.47:0/1475692736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f79377880 con 0x564f794d77d0 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f30977fe700 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f3096ffd700 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794d7bb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3088010d90 tx=0x7f3088013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b18b48279855fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f307d7fa700 1 -- 172.21.15.47:0/1475692736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308801e420 con 0x564f794d77d0 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f307d7fa700 1 -- 172.21.15.47:0/1475692736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308801ea70 con 0x564f794d77d0 2024-04-25T20:59:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.935+0000 7f307d7fa700 1 -- 172.21.15.47:0/1475692736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308801ca60 con 0x564f794d77d0 2024-04-25T20:59:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 -- 172.21.15.47:0/1475692736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 msgr2=0x564f794d7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794d7bb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3088010d90 tx=0x7f3088013000 comp rx=0 tx=0).stop 2024-04-25T20:59:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 -- 172.21.15.47:0/1475692736 shutdown_connections 2024-04-25T20:59:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794d7bb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/1475692736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 0x564f794dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 -- 172.21.15.47:0/1475692736 >> 172.21.15.47:0/1475692736 conn(0x564f79404640 msgr2=0x564f794d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 -- 172.21.15.47:0/1475692736 shutdown_connections 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 -- 172.21.15.47:0/1475692736 wait complete. 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 -- start start 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.936+0000 7f30a7b74b80 1 Processor -- start 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d77d0 0x564f794e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 0x564f794e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794dc580 0x564f794e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564f794e4610 con 0x564f794dc580 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564f793c8b80 con 0x564f794d77d0 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564f794e4790 con 0x564f794d80f0 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30a7b74b80 10 client.0 _ll_register_callbacks cb 0x564f793c2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794dc580 0x564f794e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794dc580 0x564f794e1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48264/0 (socket says 172.21.15.47:48264) 2024-04-25T20:59:07.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 -- 172.21.15.47:0/2655185286 learned_addr learned my addr 172.21.15.47:0/2655185286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f3096ffd700 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d77d0 0x564f794e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30967fc700 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 0x564f794e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 -- 172.21.15.47:0/2655185286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 msgr2=0x564f794e0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 0x564f794e0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 -- 172.21.15.47:0/2655185286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d77d0 msgr2=0x564f794e3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d77d0 0x564f794e3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 -- 172.21.15.47:0/2655185286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f793e1220 con 0x564f794dc580 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f30977fe700 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794dc580 0x564f794e1290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f30900cd8c0 tx=0x7f309009c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e893ee3cedafd556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f307aff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f307aff5700 1 -- 172.21.15.47:0/2655185286 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30900b5060 con 0x564f794dc580 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.937+0000 7f307aff5700 1 -- 172.21.15.47:0/2655185286 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f3090004d60 con 0x564f794dc580 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 1 -- 172.21.15.47:0/2655185286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f308801ea70 con 0x564f794dc580 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f307aff5700 1 -- 172.21.15.47:0/2655185286 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f309009c130 con 0x564f794dc580 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 10 client.14952 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 1 -- 172.21.15.47:0/2655185286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f794eee70 con 0x564f794dc580 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f307aff5700 1 -- 172.21.15.47:0/2655185286 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308801ea70 con 0x564f794dc580 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f307aff5700 1 -- 172.21.15.47:0/2655185286 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3090097e80 con 0x564f794dc580 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 10 client.14952 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 10 client.14952 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 20 client.14952 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x564f794f18b0 0x564f794f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 4 client.14952 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.938+0000 7f30a7b74b80 1 -- 172.21.15.47:0/2655185286 --> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f794374d0 con 0x564f794f18b0 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.939+0000 7f3096ffd700 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x564f794f18b0 0x564f794f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.939+0000 7f307aff5700 10 client.14952 ms_handle_connect on v2:172.21.15.92:6834/2097108178 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.939+0000 7f3096ffd700 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x564f794f18b0 0x564f794f2d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a03f28c2a426edd server_cookie=a76b88c86c16dc61 in_seq=0 out_seq=0 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f307aff5700 1 -- 172.21.15.47:0/2655185286 <== mds.2 v2:172.21.15.92:6834/2097108178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f794374d0 con 0x564f794f18b0 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f307aff5700 10 client.14952 handle_command_reply: tid=0 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 client.14952 shutdown 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 -- 172.21.15.47:0/2655185286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794dc580 msgr2=0x564f794e1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794dc580 0x564f794e1290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f30900cd8c0 tx=0x7f309009c670 comp rx=0 tx=0).stop 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 -- 172.21.15.47:0/2655185286 shutdown_connections 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x564f794f18b0 0x564f794f2d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794dc580 0x564f794e1290 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d77d0 0x564f794e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/2655185286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d80f0 0x564f794e0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 -- 172.21.15.47:0/2655185286 >> 172.21.15.47:0/2655185286 conn(0x564f79404640 msgr2=0x564f794dd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f307aff5700 0 client.14952 ms_handle_reset on v2:172.21.15.92:6834/2097108178 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.940+0000 7f30a7b74b80 1 -- 172.21.15.47:0/2655185286 shutdown_connections 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.941+0000 7f30a7b74b80 1 -- 172.21.15.47:0/2655185286 wait complete. 2024-04-25T20:59:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.941+0000 7f30a7b74b80 20 client.14952 trim_cache size 0 max 16384 2024-04-25T20:59:07.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:07 smithi186 ceph-mon[32529]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.972+0000 7f30a7b74b80 1 Processor -- start 2024-04-25T20:59:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.972+0000 7f30a7b74b80 1 -- start start 2024-04-25T20:59:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d80f0 0x564f794f0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794dc580 0x564f794de060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564f794f1060 con 0x564f794d77d0 2024-04-25T20:59:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564f794f1730 con 0x564f794d80f0 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564f794f1e00 con 0x564f794dc580 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794f02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48270/0 (socket says 172.21.15.47:48270) 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 -- 172.21.15.47:0/485984564 learned_addr learned my addr 172.21.15.47:0/485984564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f30967fc700 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d80f0 0x564f794f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f30977fe700 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794dc580 0x564f794de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 -- 172.21.15.47:0/485984564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794dc580 msgr2=0x564f794de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794dc580 0x564f794de060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 -- 172.21.15.47:0/485984564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d80f0 msgr2=0x564f794f0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d80f0 0x564f794f0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 -- 172.21.15.47:0/485984564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f794dc360 con 0x564f794d77d0 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.973+0000 7f3096ffd700 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794f02e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3088011ab0 tx=0x7f308801c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9422ad3210a909f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f307cff9700 1 -- 172.21.15.47:0/485984564 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088005240 con 0x564f794d77d0 2024-04-25T20:59:07.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f307cff9700 1 -- 172.21.15.47:0/485984564 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308801c490 con 0x564f794d77d0 2024-04-25T20:59:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f307cff9700 1 -- 172.21.15.47:0/485984564 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088003e60 con 0x564f794d77d0 2024-04-25T20:59:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f30a7b74b80 1 -- 172.21.15.47:0/485984564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 msgr2=0x564f794f02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794f02e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3088011ab0 tx=0x7f308801c640 comp rx=0 tx=0).stop 2024-04-25T20:59:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f30a7b74b80 1 -- 172.21.15.47:0/485984564 shutdown_connections 2024-04-25T20:59:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d77d0 0x564f794f02e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794dc580 0x564f794de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/485984564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794d80f0 0x564f794f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f30a7b74b80 1 -- 172.21.15.47:0/485984564 >> 172.21.15.47:0/485984564 conn(0x564f79404640 msgr2=0x564f794df640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.974+0000 7f30a7b74b80 1 -- 172.21.15.47:0/485984564 shutdown_connections 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 -- 172.21.15.47:0/485984564 wait complete. 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 -- start start 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 Processor -- start 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d77d0 0x564f794f9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d80f0 0x564f794f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564f794f2ba0 con 0x564f794d80f0 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564f794f24d0 con 0x564f794dc580 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564f794f3270 con 0x564f794d77d0 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30a7b74b80 10 client.0 _ll_register_callbacks cb 0x564f793c2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d80f0 0x564f794f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d80f0 0x564f794f9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48284/0 (socket says 172.21.15.47:48284) 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30967fc700 1 -- 172.21.15.47:0/515747318 learned_addr learned my addr 172.21.15.47:0/515747318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f30977fe700 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.975+0000 7f3096ffd700 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d77d0 0x564f794f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30967fc700 1 -- 172.21.15.47:0/515747318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d77d0 msgr2=0x564f794f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30967fc700 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d77d0 0x564f794f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30967fc700 1 -- 172.21.15.47:0/515747318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 msgr2=0x564f794dddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30967fc700 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30967fc700 1 -- 172.21.15.47:0/515747318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f794de2f0 con 0x564f794d80f0 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30967fc700 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d80f0 0x564f794f9e60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f308c009990 tx=0x7f308c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dab2f562cecb6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f307aff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f307aff5700 1 -- 172.21.15.47:0/515747318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00bb30 con 0x564f794d80f0 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30a7b74b80 1 -- 172.21.15.47:0/515747318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f308801c490 con 0x564f794d80f0 2024-04-25T20:59:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f307aff5700 1 -- 172.21.15.47:0/515747318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f308c00bcd0 con 0x564f794d80f0 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f307aff5700 1 -- 172.21.15.47:0/515747318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f308c034050 con 0x564f794d80f0 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30a7b74b80 10 client.14958 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f30a7b74b80 1 -- 172.21.15.47:0/515747318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f79559090 con 0x564f794d80f0 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.976+0000 7f307aff5700 1 -- 172.21.15.47:0/515747318 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308801c490 con 0x564f794d80f0 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f307aff5700 1 -- 172.21.15.47:0/515747318 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f308c00f620 con 0x564f794d80f0 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f30a7b74b80 10 client.14958 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f30a7b74b80 10 client.14958 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f30a7b74b80 20 client.14958 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x564f7955dcf0 0x564f7955ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f30a7b74b80 4 client.14958 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f30a7b74b80 1 -- 172.21.15.47:0/515747318 --> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] -- command(tid 0: {"prefix": "status"}) -- 0x564f79431570 con 0x564f7955dcf0 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f3096ffd700 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x564f7955dcf0 0x564f7955ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f3096ffd700 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x564f7955dcf0 0x564f7955ca00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc47657c15952bdf server_cookie=e2c0bfbdadd92001 in_seq=0 out_seq=0 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.977+0000 7f307aff5700 10 client.14958 ms_handle_connect on v2:172.21.15.92:6834/2097108178 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f307aff5700 1 -- 172.21.15.47:0/515747318 <== mds.2 v2:172.21.15.92:6834/2097108178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564f79431570 con 0x564f7955dcf0 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f307aff5700 10 client.14958 handle_command_reply: tid=0 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 client.14958 shutdown 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 -- 172.21.15.47:0/515747318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d80f0 msgr2=0x564f794f9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d80f0 0x564f794f9e60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f308c009990 tx=0x7f308c005790 comp rx=0 tx=0).stop 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 -- 172.21.15.47:0/515747318 shutdown_connections 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x564f7955dcf0 0x564f7955ca00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f794dc580 0x564f794dddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f794d77d0 0x564f794f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 --2- 172.21.15.47:0/515747318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f794d80f0 0x564f794f9e60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 -- 172.21.15.47:0/515747318 >> 172.21.15.47:0/515747318 conn(0x564f79404640 msgr2=0x564f794df4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f307aff5700 0 client.14958 ms_handle_reset on v2:172.21.15.92:6834/2097108178 2024-04-25T20:59:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 -- 172.21.15.47:0/515747318 shutdown_connections 2024-04-25T20:59:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 1 -- 172.21.15.47:0/515747318 wait complete. 2024-04-25T20:59:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f30a7b74b80 20 client.14958 trim_cache size 0 max 16384 2024-04-25T20:59:07.984 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T20:59:07.985 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:59:07.985 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-04-25T20:59:07.985 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24452, 2024-04-25T20:59:07.985 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T20:59:07.985 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T20:59:07.985 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T20:59:07.985 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 121.26382304000001, 2024-04-25T20:59:07.986 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 25, 2024-04-25T20:59:07.986 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 91, 2024-04-25T20:59:07.986 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T20:59:07.986 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 171.39379375999999 2024-04-25T20:59:07.986 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f309d2eb700 1 -- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f30800a3cc0 msgr2=0x7f30800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f309d2eb700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f30800a3cc0 0x7f30800a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3088007b60 tx=0x7f30880073d0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f309d2eb700 1 -- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3098108a60 msgr2=0x7f3098102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.978+0000 7f309d2eb700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3098108a60 0x7f3098102430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f309000ef30 tx=0x7f309000c3a0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.979+0000 7f309d2eb700 1 -- 172.21.15.47:0/2702915012 shutdown_connections 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.979+0000 7f309d2eb700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f30800a3cc0 0x7f30800a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.979+0000 7f309d2eb700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3098108a60 0x7f3098102430 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.979+0000 7f309d2eb700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309806b1f0 0x7f30981019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.979+0000 7f309d2eb700 1 --2- 172.21.15.47:0/2702915012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f309806bbb0 0x7f3098101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.979+0000 7f309d2eb700 1 -- 172.21.15.47:0/2702915012 >> 172.21.15.47:0/2702915012 conn(0x7f30980789e0 msgr2=0x7f30980ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.979+0000 7f309d2eb700 1 -- 172.21.15.47:0/2702915012 shutdown_connections 2024-04-25T20:59:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:07.979+0000 7f309d2eb700 1 -- 172.21.15.47:0/2702915012 wait complete. 2024-04-25T20:59:08.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:07 smithi092 ceph-mon[32524]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:08.689 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T20:59:08.981 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:09 smithi047 ceph-mon[29195]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:09.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:09 smithi186 ceph-mon[32529]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:10.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:09 smithi092 ceph-mon[32524]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.594+0000 7f98a7af5700 1 -- 172.21.15.47:0/2117143404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 msgr2=0x7f98a010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.594+0000 7f98a7af5700 1 --2- 172.21.15.47:0/2117143404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 0x7f98a010f8b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f989800b0a0 tx=0x7f989800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.594+0000 7f98a7af5700 1 -- 172.21.15.47:0/2117143404 shutdown_connections 2024-04-25T20:59:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 --2- 172.21.15.47:0/2117143404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 0x7f98a010f8b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 --2- 172.21.15.47:0/2117143404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a0108a50 0x7f98a0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 --2- 172.21.15.47:0/2117143404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a0102a90 0x7f98a0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 -- 172.21.15.47:0/2117143404 >> 172.21.15.47:0/2117143404 conn(0x7f98a0079ba0 msgr2=0x7f98a0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 -- 172.21.15.47:0/2117143404 shutdown_connections 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 -- 172.21.15.47:0/2117143404 wait complete. 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 Processor -- start 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 -- start start 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.595+0000 7f98a7af5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a0102a90 0x7f98a0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a7af5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 0x7f98a0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a7af5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a0108a50 0x7f98a00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a7af5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0112c80 con 0x7f98a0103450 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a7af5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0112b00 con 0x7f98a0102a90 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a7af5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0112e00 con 0x7f98a0108a50 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 0x7f98a0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 0x7f98a0075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48294/0 (socket says 172.21.15.47:48294) 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 -- 172.21.15.47:0/3389651696 learned_addr learned my addr 172.21.15.47:0/3389651696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5891700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a0102a90 0x7f98a0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a6092700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a0108a50 0x7f98a00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 -- 172.21.15.47:0/3389651696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a0108a50 msgr2=0x7f98a00757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a0108a50 0x7f98a00757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 -- 172.21.15.47:0/3389651696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a0102a90 msgr2=0x7f98a0074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a0102a90 0x7f98a0074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 -- 172.21.15.47:0/3389651696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a01a76a0 con 0x7f98a0103450 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a5090700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 0x7f98a0075280 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f989c00b530 tx=0x7f989c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f9892ffd700 1 -- 172.21.15.47:0/3389651696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c015020 con 0x7f98a0103450 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f9892ffd700 1 -- 172.21.15.47:0/3389651696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c011ce0 con 0x7f98a0103450 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a7af5700 1 -- 172.21.15.47:0/3389651696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a01a7990 con 0x7f98a0103450 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f9892ffd700 1 -- 172.21.15.47:0/3389651696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c010dd0 con 0x7f98a0103450 2024-04-25T20:59:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.596+0000 7f98a7af5700 1 -- 172.21.15.47:0/3389651696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a01a7ed0 con 0x7f98a0103450 2024-04-25T20:59:10.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.599+0000 7f9892ffd700 1 -- 172.21.15.47:0/3389651696 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f989c018020 con 0x7f98a0103450 2024-04-25T20:59:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.599+0000 7f9892ffd700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f988c0a3c40 0x7f988c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.599+0000 7f98a5891700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f988c0a3c40 0x7f988c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.599+0000 7f9892ffd700 1 -- 172.21.15.47:0/3389651696 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f989c0cfae0 con 0x7f98a0103450 2024-04-25T20:59:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.600+0000 7f98a5891700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f988c0a3c40 0x7f988c0a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9894005950 tx=0x7f989400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.600+0000 7f98b237eb80 1 Processor -- start 2024-04-25T20:59:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.600+0000 7f98b237eb80 1 -- start start 2024-04-25T20:59:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.600+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4e7d0 0x55bf51e4ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.600+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e53040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.600+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.600+0000 7f98b237eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf51da0ce0 con 0x55bf51e4f0f0 2024-04-25T20:59:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98b237eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf51d47910 con 0x55bf51e53580 2024-04-25T20:59:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98b237eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf51d9cca0 con 0x55bf51e4e7d0 2024-04-25T20:59:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e53040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e53040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48310/0 (socket says 172.21.15.47:48310) 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 -- 172.21.15.47:0/1711633713 learned_addr learned my addr 172.21.15.47:0/1711633713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5891700 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4e7d0 0x55bf51e4ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a6092700 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 -- 172.21.15.47:0/1711633713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4e7d0 msgr2=0x55bf51e4ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4e7d0 0x55bf51e4ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 -- 172.21.15.47:0/1711633713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 msgr2=0x55bf51e539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 -- 172.21.15.47:0/1711633713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf51cee880 con 0x55bf51e4f0f0 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f98a5090700 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e53040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f989c09bd40 tx=0x7f989c010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1708bb7ea0784758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f988bfff700 1 -- 172.21.15.47:0/1711633713 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c0b5070 con 0x55bf51e4f0f0 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f988bfff700 1 -- 172.21.15.47:0/1711633713 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c09f030 con 0x55bf51e4f0f0 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.601+0000 7f988bfff700 1 -- 172.21.15.47:0/1711633713 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c0b5210 con 0x55bf51e4f0f0 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 -- 172.21.15.47:0/1711633713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 msgr2=0x55bf51e53040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e53040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f989c09bd40 tx=0x7f989c010870 comp rx=0 tx=0).stop 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 -- 172.21.15.47:0/1711633713 shutdown_connections 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4e7d0 0x55bf51e4ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1711633713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e53040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 -- 172.21.15.47:0/1711633713 >> 172.21.15.47:0/1711633713 conn(0x55bf51d7b640 msgr2=0x55bf51e49820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 -- 172.21.15.47:0/1711633713 shutdown_connections 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 -- 172.21.15.47:0/1711633713 wait complete. 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.602+0000 7f98b237eb80 1 -- start start 2024-04-25T20:59:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98b237eb80 1 Processor -- start 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4e7d0 0x55bf51e5aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4f0f0 0x55bf51e57da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98b237eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf51e5b610 con 0x55bf51e4e7d0 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98b237eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf51d3fb80 con 0x55bf51e53580 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98b237eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf51e5b790 con 0x55bf51e4f0f0 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98b237eb80 10 client.0 _ll_register_callbacks cb 0x55bf51d39900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98a5891700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4e7d0 0x55bf51e5aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98a5891700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4e7d0 0x55bf51e5aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48314/0 (socket says 172.21.15.47:48314) 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98a5891700 1 -- 172.21.15.47:0/1512814940 learned_addr learned my addr 172.21.15.47:0/1512814940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98a5090700 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4f0f0 0x55bf51e57da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.603+0000 7f98a6092700 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98a5891700 1 -- 172.21.15.47:0/1512814940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4f0f0 msgr2=0x55bf51e57da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98a5891700 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4f0f0 0x55bf51e57da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98a5891700 1 -- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 msgr2=0x55bf51e582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98a5891700 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98a5891700 1 -- 172.21.15.47:0/1512814940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf51d58220 con 0x55bf51e4e7d0 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98a6092700 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98a5891700 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4e7d0 0x55bf51e5aaa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f98940116c0 tx=0x7f98940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368b3de304ebc7dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98897fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98897fa700 1 -- 172.21.15.47:0/1512814940 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894004a30 con 0x55bf51e4e7d0 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98b237eb80 1 -- 172.21.15.47:0/1512814940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f989c09f030 con 0x55bf51e4e7d0 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98897fa700 1 -- 172.21.15.47:0/1512814940 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f9894004bd0 con 0x55bf51e4e7d0 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98897fa700 1 -- 172.21.15.47:0/1512814940 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9894010e80 con 0x55bf51e4e7d0 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98b237eb80 10 client.14970 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98b237eb80 1 -- 172.21.15.47:0/1512814940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf51e65f70 con 0x55bf51e4e7d0 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.604+0000 7f98897fa700 1 -- 172.21.15.47:0/1512814940 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c09f030 con 0x55bf51e4e7d0 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.605+0000 7f98897fa700 1 -- 172.21.15.47:0/1512814940 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f989401f890 con 0x55bf51e4e7d0 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.605+0000 7f98b237eb80 10 client.14970 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.605+0000 7f98b237eb80 10 client.14970 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.605+0000 7f98b237eb80 20 client.14970 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.605+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55bf51e68550 0x55bf51e699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.605+0000 7f98b237eb80 4 client.14970 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.605+0000 7f98b237eb80 1 -- 172.21.15.47:0/1512814940 --> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf51dae4d0 con 0x55bf51e68550 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.605+0000 7f98a5090700 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55bf51e68550 0x55bf51e699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.606+0000 7f98a5090700 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55bf51e68550 0x55bf51e699c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc81c5fde5d703f9 server_cookie=9215b9c2d858dcc2 in_seq=0 out_seq=0 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.606+0000 7f98897fa700 10 client.14970 ms_handle_connect on v2:172.21.15.92:6836/1984144918 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.606+0000 7f98897fa700 1 -- 172.21.15.47:0/1512814940 <== mds.1 v2:172.21.15.92:6836/1984144918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf51dae4d0 con 0x55bf51e68550 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.606+0000 7f98897fa700 10 client.14970 handle_command_reply: tid=0 2024-04-25T20:59:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.606+0000 7f98b237eb80 1 client.14970 shutdown 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.606+0000 7f98b237eb80 1 -- 172.21.15.47:0/1512814940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4e7d0 msgr2=0x55bf51e5aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.606+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4e7d0 0x55bf51e5aaa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f98940116c0 tx=0x7f98940047c0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 1 -- 172.21.15.47:0/1512814940 shutdown_connections 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55bf51e68550 0x55bf51e699c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4e7d0 0x55bf51e5aaa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e53580 0x55bf51e582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98897fa700 0 client.14970 ms_handle_reset on v2:172.21.15.92:6836/1984144918 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1512814940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e4f0f0 0x55bf51e57da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 1 -- 172.21.15.47:0/1512814940 >> 172.21.15.47:0/1512814940 conn(0x55bf51d7b640 msgr2=0x55bf51e48a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 1 -- 172.21.15.47:0/1512814940 shutdown_connections 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 1 -- 172.21.15.47:0/1512814940 wait complete. 2024-04-25T20:59:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.607+0000 7f98b237eb80 20 client.14970 trim_cache size 0 max 16384 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.627+0000 7f98b237eb80 1 Processor -- start 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.627+0000 7f98b237eb80 1 -- start start 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.627+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e56d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.627+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e5a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.627+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e53580 0x55bf51e52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.627+0000 7f98b237eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf51e67d00 con 0x55bf51e4f0f0 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.627+0000 7f98b237eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf51e683d0 con 0x55bf51e4e7d0 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.627+0000 7f98b237eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf51e68aa0 con 0x55bf51e53580 2024-04-25T20:59:10.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e5a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e5a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48328/0 (socket says 172.21.15.47:48328) 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 -- 172.21.15.47:0/1249159789 learned_addr learned my addr 172.21.15.47:0/1249159789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5891700 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e56d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a6092700 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e53580 0x55bf51e52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 -- 172.21.15.47:0/1249159789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e53580 msgr2=0x55bf51e52580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e53580 0x55bf51e52580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 -- 172.21.15.47:0/1249159789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 msgr2=0x55bf51e56d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e56d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 -- 172.21.15.47:0/1249159789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf51e56310 con 0x55bf51e4f0f0 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5891700 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e56d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f98a5090700 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e5a5e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f989c098780 tx=0x7f989c0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d52e127432d602f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f988b7fe700 1 -- 172.21.15.47:0/1249159789 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c0b5040 con 0x55bf51e4f0f0 2024-04-25T20:59:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f988b7fe700 1 -- 172.21.15.47:0/1249159789 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c0a0160 con 0x55bf51e4f0f0 2024-04-25T20:59:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.628+0000 7f988b7fe700 1 -- 172.21.15.47:0/1249159789 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c0a4590 con 0x55bf51e4f0f0 2024-04-25T20:59:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 -- 172.21.15.47:0/1249159789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 msgr2=0x55bf51e5a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e5a5e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f989c098780 tx=0x7f989c0bc040 comp rx=0 tx=0).stop 2024-04-25T20:59:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 -- 172.21.15.47:0/1249159789 shutdown_connections 2024-04-25T20:59:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e53580 0x55bf51e52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e56d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 --2- 172.21.15.47:0/1249159789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e4f0f0 0x55bf51e5a5e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 -- 172.21.15.47:0/1249159789 >> 172.21.15.47:0/1249159789 conn(0x55bf51d7b640 msgr2=0x55bf51e51e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 -- 172.21.15.47:0/1249159789 shutdown_connections 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 -- 172.21.15.47:0/1249159789 wait complete. 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 -- start start 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.629+0000 7f98b237eb80 1 Processor -- start 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e53580 0x55bf51e555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98b237eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e713e0 0x55bf51e71860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98b237eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf51e69840 con 0x55bf51e53580 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98b237eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf51e69170 con 0x55bf51e4e7d0 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98b237eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf51e69f10 con 0x55bf51e713e0 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98b237eb80 10 client.0 _ll_register_callbacks cb 0x55bf51d39900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a6092700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e713e0 0x55bf51e71860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a5090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e53580 0x55bf51e555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a6092700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e713e0 0x55bf51e71860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:40628/0 (socket says 172.21.15.47:40628) 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a6092700 1 -- 172.21.15.47:0/2710017458 learned_addr learned my addr 172.21.15.47:0/2710017458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a5090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e53580 0x55bf51e555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48332/0 (socket says 172.21.15.47:48332) 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a5891700 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a6092700 1 -- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 msgr2=0x55bf51e550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a6092700 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a6092700 1 -- 172.21.15.47:0/2710017458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e53580 msgr2=0x55bf51e555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a6092700 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e53580 0x55bf51e555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a6092700 1 -- 172.21.15.47:0/2710017458 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf51e71da0 con 0x55bf51e713e0 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a5891700 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.630+0000 7f98a5090700 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e53580 0x55bf51e555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98a6092700 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e713e0 0x55bf51e71860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9898003d70 tx=0x7f9898003da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a753a6e82c99afe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98897fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-25T20:59:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98897fa700 1 -- 172.21.15.47:0/2710017458 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98980077b0 con 0x55bf51e713e0 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98b237eb80 1 -- 172.21.15.47:0/2710017458 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f989c0a0160 con 0x55bf51e713e0 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98897fa700 1 -- 172.21.15.47:0/2710017458 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f9898007950 con 0x55bf51e713e0 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98897fa700 1 -- 172.21.15.47:0/2710017458 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9898007bb0 con 0x55bf51e713e0 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98b237eb80 10 client.24709 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98b237eb80 1 -- 172.21.15.47:0/2710017458 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf51ed0250 con 0x55bf51e713e0 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.631+0000 7f98897fa700 1 -- 172.21.15.47:0/2710017458 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c0a0160 con 0x55bf51e713e0 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98897fa700 1 -- 172.21.15.47:0/2710017458 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9898011070 con 0x55bf51e713e0 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98b237eb80 10 client.24709 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98b237eb80 10 client.24709 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98b237eb80 20 client.24709 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98b237eb80 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55bf51ed4e10 0x55bf51ed3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98b237eb80 4 client.24709 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98b237eb80 1 -- 172.21.15.47:0/2710017458 --> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] -- command(tid 0: {"prefix": "status"}) -- 0x55bf51da8570 con 0x55bf51ed4e10 2024-04-25T20:59:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98a5891700 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55bf51ed4e10 0x55bf51ed3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98a5891700 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55bf51ed4e10 0x55bf51ed3b20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ec5e36a28939d81 server_cookie=4da804bcecff6be3 in_seq=0 out_seq=0 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.632+0000 7f98897fa700 10 client.24709 ms_handle_connect on v2:172.21.15.92:6836/1984144918 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98897fa700 1 -- 172.21.15.47:0/2710017458 <== mds.1 v2:172.21.15.92:6836/1984144918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bf51da8570 con 0x55bf51ed4e10 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98897fa700 10 client.24709 handle_command_reply: tid=0 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 client.24709 shutdown 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 -- 172.21.15.47:0/2710017458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e713e0 msgr2=0x55bf51e71860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e713e0 0x55bf51e71860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9898003d70 tx=0x7f9898003da0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 -- 172.21.15.47:0/2710017458 shutdown_connections 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55bf51ed4e10 0x55bf51ed3b20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf51e713e0 0x55bf51e71860 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf51e4e7d0 0x55bf51e550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98897fa700 0 client.24709 ms_handle_reset on v2:172.21.15.92:6836/1984144918 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 --2- 172.21.15.47:0/2710017458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf51e53580 0x55bf51e555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 -- 172.21.15.47:0/2710017458 >> 172.21.15.47:0/2710017458 conn(0x55bf51d7b640 msgr2=0x55bf51e52700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 -- 172.21.15.47:0/2710017458 shutdown_connections 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 1 -- 172.21.15.47:0/2710017458 wait complete. 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.633+0000 7f98b237eb80 20 client.24709 trim_cache size 0 max 16384 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24467, 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T20:59:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 124.930274692, 2024-04-25T20:59:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 25, 2024-04-25T20:59:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 91, 2024-04-25T20:59:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T20:59:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 165.70588211899999 2024-04-25T20:59:10.640 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T20:59:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 -- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f988c0a3c40 msgr2=0x7f988c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f988c0a3c40 0x7f988c0a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9894005950 tx=0x7f989400b410 comp rx=0 tx=0).stop 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 -- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 msgr2=0x7f98a0075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 0x7f98a0075280 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f989c00b530 tx=0x7f989c00b840 comp rx=0 tx=0).stop 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 -- 172.21.15.47:0/3389651696 shutdown_connections 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f988c0a3c40 0x7f988c0a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a0108a50 0x7f98a00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a0102a90 0x7f98a0074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 --2- 172.21.15.47:0/3389651696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a0103450 0x7f98a0075280 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 -- 172.21.15.47:0/3389651696 >> 172.21.15.47:0/3389651696 conn(0x7f98a0079ba0 msgr2=0x7f98a01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 -- 172.21.15.47:0/3389651696 shutdown_connections 2024-04-25T20:59:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:10.634+0000 7f98a7af5700 1 -- 172.21.15.47:0/3389651696 wait complete. 2024-04-25T20:59:11.359 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T20:59:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:11.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:11 smithi047 ceph-mon[29195]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:11.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:11 smithi186 ceph-mon[32529]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:12.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:11 smithi092 ceph-mon[32524]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:13.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.281+0000 7ff8f90a3700 1 -- 172.21.15.47:0/544899239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4102a90 msgr2=0x7ff8f4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.281+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/544899239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4102a90 0x7ff8f4102f10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009990 tx=0x7ff8e8009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.282+0000 7ff8f90a3700 1 -- 172.21.15.47:0/544899239 shutdown_connections 2024-04-25T20:59:13.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.282+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/544899239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f4103450 0x7ff8f410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.282+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/544899239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f4108a50 0x7ff8f4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.282+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/544899239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4102a90 0x7ff8f4102f10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.282+0000 7ff8f90a3700 1 -- 172.21.15.47:0/544899239 >> 172.21.15.47:0/544899239 conn(0x7ff8f4079ba0 msgr2=0x7ff8f4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.282+0000 7ff8f90a3700 1 -- 172.21.15.47:0/544899239 shutdown_connections 2024-04-25T20:59:13.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.282+0000 7ff8f90a3700 1 -- 172.21.15.47:0/544899239 wait complete. 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.282+0000 7ff8f90a3700 1 Processor -- start 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f90a3700 1 -- start start 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f90a3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f4102a90 0x7ff8f4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f90a3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4103450 0x7ff8f41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f90a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f4108a50 0x7ff8f419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f90a3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4112c80 con 0x7ff8f4103450 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f90a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4112b00 con 0x7ff8f4108a50 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f90a3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4112e00 con 0x7ff8f4102a90 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4103450 0x7ff8f41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4103450 0x7ff8f41996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48352/0 (socket says 172.21.15.47:48352) 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f259c700 1 -- 172.21.15.47:0/4085267601 learned_addr learned my addr 172.21.15.47:0/4085267601 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f4102a90 0x7ff8f4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f359e700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f4108a50 0x7ff8f419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f259c700 1 -- 172.21.15.47:0/4085267601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f4102a90 msgr2=0x7ff8f4199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f259c700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f4102a90 0x7ff8f4199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f259c700 1 -- 172.21.15.47:0/4085267601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f4108a50 msgr2=0x7ff8f419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f259c700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f4108a50 0x7ff8f419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.283+0000 7ff8f259c700 1 -- 172.21.15.47:0/4085267601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f419e180 con 0x7ff8f4103450 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.284+0000 7ff8f259c700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4103450 0x7ff8f41996b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8000c00 tx=0x7ff8e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.284+0000 7ff8dbfff700 1 -- 172.21.15.47:0/4085267601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8027070 con 0x7ff8f4103450 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.284+0000 7ff8dbfff700 1 -- 172.21.15.47:0/4085267601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e80040e0 con 0x7ff8f4103450 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.284+0000 7ff8f90a3700 1 -- 172.21.15.47:0/4085267601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f419e410 con 0x7ff8f4103450 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.284+0000 7ff8dbfff700 1 -- 172.21.15.47:0/4085267601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800f460 con 0x7ff8f4103450 2024-04-25T20:59:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.284+0000 7ff8f90a3700 1 -- 172.21.15.47:0/4085267601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41a5cf0 con 0x7ff8f4103450 2024-04-25T20:59:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.286+0000 7ff8dbfff700 1 -- 172.21.15.47:0/4085267601 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff8e802a070 con 0x7ff8f4103450 2024-04-25T20:59:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.287+0000 7ff8dbfff700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8dc0a3c40 0x7ff8dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.287+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8dc0a3c40 0x7ff8dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.287+0000 7ff8dbfff700 1 -- 172.21.15.47:0/4085267601 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7ff8e80d13d0 con 0x7ff8f4103450 2024-04-25T20:59:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.287+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8dc0a3c40 0x7ff8dc0a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005950 tx=0x7ff8e400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.287+0000 7ff90392cb80 1 Processor -- start 2024-04-25T20:59:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.287+0000 7ff90392cb80 1 -- start start 2024-04-25T20:59:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 0x562ab131c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1327340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1329c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff90392cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562ab126fc40 con 0x562ab1327880 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff90392cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562ab12168e0 con 0x562ab131c0b0 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff90392cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562ab126bc00 con 0x562ab131c9d0 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1329c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1329c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48358/0 (socket says 172.21.15.47:48358) 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 -- 172.21.15.47:0/373317063 learned_addr learned my addr 172.21.15.47:0/373317063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 0x562ab131c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f259c700 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1327340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 -- 172.21.15.47:0/373317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 msgr2=0x562ab1327340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1327340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 -- 172.21.15.47:0/373317063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 msgr2=0x562ab131c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 0x562ab131c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 -- 172.21.15.47:0/373317063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ab11da6f0 con 0x562ab1327880 2024-04-25T20:59:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8f359e700 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1329c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009f20 tx=0x7ff8ec009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e562d6ede4a649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8d97fa700 1 -- 172.21.15.47:0/373317063 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec01a070 con 0x562ab1327880 2024-04-25T20:59:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.288+0000 7ff8d97fa700 1 -- 172.21.15.47:0/373317063 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec015410 con 0x562ab1327880 2024-04-25T20:59:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff8d97fa700 1 -- 172.21.15.47:0/373317063 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec00a660 con 0x562ab1327880 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 -- 172.21.15.47:0/373317063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 msgr2=0x562ab1329c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1329c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009f20 tx=0x7ff8ec009f50 comp rx=0 tx=0).stop 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 -- 172.21.15.47:0/373317063 shutdown_connections 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1329c70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 0x562ab131c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 --2- 172.21.15.47:0/373317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1327340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 -- 172.21.15.47:0/373317063 >> 172.21.15.47:0/373317063 conn(0x562ab1249840 msgr2=0x562ab1318580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 -- 172.21.15.47:0/373317063 shutdown_connections 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.289+0000 7ff90392cb80 1 -- 172.21.15.47:0/373317063 wait complete. 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 1 -- start start 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 1 Processor -- start 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab131c0b0 0x562ab1327010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1324310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab1327880 0x562ab1324850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562ab132a370 con 0x562ab131c0b0 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562ab1229560 con 0x562ab1327880 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562ab132a4f0 con 0x562ab131c9d0 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff90392cb80 10 client.0 _ll_register_callbacks cb 0x562ab12088d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff8f2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab131c0b0 0x562ab1327010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff8f2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab131c0b0 0x562ab1327010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48374/0 (socket says 172.21.15.47:48374) 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff8f2d9d700 1 -- 172.21.15.47:0/2390074783 learned_addr learned my addr 172.21.15.47:0/2390074783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff8f259c700 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1324310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.290+0000 7ff8f359e700 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab1327880 0x562ab1324850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8f2d9d700 1 -- 172.21.15.47:0/2390074783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 msgr2=0x562ab1324310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1324310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8f2d9d700 1 -- 172.21.15.47:0/2390074783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab1327880 msgr2=0x562ab1324850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab1327880 0x562ab1324850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8f2d9d700 1 -- 172.21.15.47:0/2390074783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ab11bd880 con 0x562ab131c0b0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8f259c700 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1324310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab131c0b0 0x562ab1327010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff8e401da70 tx=0x7ff8e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee03d084c0fbe87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2390074783 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4016070 con 0x562ab131c0b0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2390074783 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7ff8e4010900 con 0x562ab131c0b0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff90392cb80 1 -- 172.21.15.47:0/2390074783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ec015410 con 0x562ab131c0b0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2390074783 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8e4010b40 con 0x562ab131c0b0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff90392cb80 10 client.14988 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff90392cb80 1 -- 172.21.15.47:0/2390074783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562ab1334bb0 con 0x562ab131c0b0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.291+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2390074783 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec015410 con 0x562ab131c0b0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2390074783 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff8e401b420 con 0x562ab131c0b0 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff90392cb80 10 client.14988 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff90392cb80 10 client.14988 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff90392cb80 20 client.14988 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562ab1337250 0x562ab13386c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff90392cb80 4 client.14988 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff90392cb80 1 -- 172.21.15.47:0/2390074783 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ab127d430 con 0x562ab1337250 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff8f259c700 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562ab1337250 0x562ab13386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff8f259c700 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562ab1337250 0x562ab13386c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6071f5e231c56480 server_cookie=fe64c0239775cd7d in_seq=0 out_seq=0 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.292+0000 7ff8d6ff5700 10 client.14988 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2390074783 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ab127d430 con 0x562ab1337250 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff8d6ff5700 10 client.14988 handle_command_reply: tid=0 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 client.14988 shutdown 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 -- 172.21.15.47:0/2390074783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab131c0b0 msgr2=0x562ab1327010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab131c0b0 0x562ab1327010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff8e401da70 tx=0x7ff8e40047c0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 -- 172.21.15.47:0/2390074783 shutdown_connections 2024-04-25T20:59:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562ab1337250 0x562ab13386c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab131c0b0 0x562ab1327010 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab1327880 0x562ab1324850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2390074783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1324310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff8d6ff5700 0 client.14988 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T20:59:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.293+0000 7ff90392cb80 1 -- 172.21.15.47:0/2390074783 >> 172.21.15.47:0/2390074783 conn(0x562ab1249840 msgr2=0x562ab13181b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.294+0000 7ff90392cb80 1 -- 172.21.15.47:0/2390074783 shutdown_connections 2024-04-25T20:59:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.294+0000 7ff90392cb80 1 -- 172.21.15.47:0/2390074783 wait complete. 2024-04-25T20:59:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.294+0000 7ff90392cb80 20 client.14988 trim_cache size 0 max 16384 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff90392cb80 1 Processor -- start 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff90392cb80 1 -- start start 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 0x562ab1326b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1341b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1342060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff90392cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562ab1336a00 con 0x562ab1327880 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff90392cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562ab13370d0 con 0x562ab131c0b0 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff90392cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562ab13377a0 con 0x562ab131c9d0 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff8f359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1342060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff8f359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1342060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48376/0 (socket says 172.21.15.47:48376) 2024-04-25T20:59:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.314+0000 7ff8f359e700 1 -- 172.21.15.47:0/359232311 learned_addr learned my addr 172.21.15.47:0/359232311 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8f259c700 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1341b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 0x562ab1326b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8f359e700 1 -- 172.21.15.47:0/359232311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 msgr2=0x562ab1341b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8f359e700 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1341b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8f359e700 1 -- 172.21.15.47:0/359232311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 msgr2=0x562ab1326b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8f359e700 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 0x562ab1326b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8f359e700 1 -- 172.21.15.47:0/359232311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ab131df20 con 0x562ab1327880 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8f359e700 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1342060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x562ab131d270 tx=0x7ff8ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d70df8e05070b5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8d8ff9700 1 -- 172.21.15.47:0/359232311 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec01a040 con 0x562ab1327880 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8d8ff9700 1 -- 172.21.15.47:0/359232311 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec0158c0 con 0x562ab1327880 2024-04-25T20:59:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.315+0000 7ff8d8ff9700 1 -- 172.21.15.47:0/359232311 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec01f680 con 0x562ab1327880 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- 172.21.15.47:0/359232311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 msgr2=0x562ab1342060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1342060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x562ab131d270 tx=0x7ff8ec004ad0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- 172.21.15.47:0/359232311 shutdown_connections 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab1342060 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c0b0 0x562ab1326b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 --2- 172.21.15.47:0/359232311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c9d0 0x562ab1341b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- 172.21.15.47:0/359232311 >> 172.21.15.47:0/359232311 conn(0x562ab1249840 msgr2=0x562ab1322cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- 172.21.15.47:0/359232311 shutdown_connections 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- 172.21.15.47:0/359232311 wait complete. 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- start start 2024-04-25T20:59:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 Processor -- start 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c0b0 0x562ab1341b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c9d0 0x562ab1342080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab133caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562ab1338540 con 0x562ab1327880 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562ab1337e70 con 0x562ab131c9d0 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562ab1338c10 con 0x562ab131c0b0 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff90392cb80 10 client.0 _ll_register_callbacks cb 0x562ab12088d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.316+0000 7ff8f359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab133caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab133caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48392/0 (socket says 172.21.15.47:48392) 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f359e700 1 -- 172.21.15.47:0/2878521568 learned_addr learned my addr 172.21.15.47:0/2878521568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c0b0 0x562ab1341b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f259c700 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c9d0 0x562ab1342080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f359e700 1 -- 172.21.15.47:0/2878521568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c0b0 msgr2=0x562ab1341b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f359e700 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c0b0 0x562ab1341b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f359e700 1 -- 172.21.15.47:0/2878521568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c9d0 msgr2=0x562ab1342080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f359e700 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c9d0 0x562ab1342080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f359e700 1 -- 172.21.15.47:0/2878521568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ab1325280 con 0x562ab1327880 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8f359e700 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab133caf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00ca00 tx=0x7ff8ec00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c68eda6682c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2878521568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec02f070 con 0x562ab1327880 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2878521568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7ff8ec01f660 con 0x562ab1327880 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff90392cb80 1 -- 172.21.15.47:0/2878521568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ec0158c0 con 0x562ab1327880 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2878521568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8ec01a040 con 0x562ab1327880 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff90392cb80 10 client.14994 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.317+0000 7ff90392cb80 1 -- 172.21.15.47:0/2878521568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562ab13192a0 con 0x562ab1327880 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2878521568 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec0158c0 con 0x562ab1327880 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2878521568 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff8ec00a7d0 con 0x562ab1327880 2024-04-25T20:59:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff90392cb80 10 client.14994 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff90392cb80 10 client.14994 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff90392cb80 20 client.14994 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562ab13a2ee0 0x562ab13a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff90392cb80 4 client.14994 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff90392cb80 1 -- 172.21.15.47:0/2878521568 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "status"}) -- 0x562ab12774d0 con 0x562ab13a2ee0 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562ab13a2ee0 0x562ab13a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff8d6ff5700 10 client.14994 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.318+0000 7ff8f2d9d700 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562ab13a2ee0 0x562ab13a1bf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295d1cb29a9a50d3 server_cookie=c265e0d95385619e in_seq=0 out_seq=0 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff8d6ff5700 1 -- 172.21.15.47:0/2878521568 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562ab12774d0 con 0x562ab13a2ee0 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff8d6ff5700 10 client.14994 handle_command_reply: tid=0 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 client.14994 shutdown 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 -- 172.21.15.47:0/2878521568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 msgr2=0x562ab133caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab133caf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00ca00 tx=0x7ff8ec00ca30 comp rx=0 tx=0).stop 2024-04-25T20:59:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 -- 172.21.15.47:0/2878521568 shutdown_connections 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562ab13a2ee0 0x562ab13a1bf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ab1327880 0x562ab133caf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ab131c0b0 0x562ab1341b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 --2- 172.21.15.47:0/2878521568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab131c9d0 0x562ab1342080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff8d6ff5700 0 client.14994 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 -- 172.21.15.47:0/2878521568 >> 172.21.15.47:0/2878521568 conn(0x562ab1249840 msgr2=0x562ab131f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 -- 172.21.15.47:0/2878521568 shutdown_connections 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 1 -- 172.21.15.47:0/2878521568 wait complete. 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.319+0000 7ff90392cb80 20 client.14994 trim_cache size 0 max 16384 2024-04-25T20:59:13.326 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24488, 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 140.043511309, 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 25, 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 91, 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 156.023522286 2024-04-25T20:59:13.327 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T20:59:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 -- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8dc0a3c40 msgr2=0x7ff8dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8dc0a3c40 0x7ff8dc0a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005950 tx=0x7ff8e400a300 comp rx=0 tx=0).stop 2024-04-25T20:59:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 -- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4103450 msgr2=0x7ff8f41996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4103450 0x7ff8f41996b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8000c00 tx=0x7ff8e80047b0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 -- 172.21.15.47:0/4085267601 shutdown_connections 2024-04-25T20:59:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff8dc0a3c40 0x7ff8dc0a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f4108a50 0x7ff8f419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f4102a90 0x7ff8f4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 --2- 172.21.15.47:0/4085267601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f4103450 0x7ff8f41996b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 -- 172.21.15.47:0/4085267601 >> 172.21.15.47:0/4085267601 conn(0x7ff8f4079ba0 msgr2=0x7ff8f4101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 -- 172.21.15.47:0/4085267601 shutdown_connections 2024-04-25T20:59:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:13.320+0000 7ff8f90a3700 1 -- 172.21.15.47:0/4085267601 wait complete. 2024-04-25T20:59:13.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:13 smithi047 ceph-mon[29195]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:13.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:13 smithi186 ceph-mon[32529]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:13.998 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T20:59:13.998 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T20:59:14.011 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T20:59:14.011 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T20:59:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:13 smithi092 ceph-mon[32524]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:14.263 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:15 smithi047 ceph-mon[29195]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 -- 172.21.15.47:0/4087387900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c105e60 msgr2=0x7f6a2c108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 --2- 172.21.15.47:0/4087387900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c105e60 0x7f6a2c108250 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009990 tx=0x7f6a1c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 -- 172.21.15.47:0/4087387900 shutdown_connections 2024-04-25T20:59:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 --2- 172.21.15.47:0/4087387900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a2c108790 0x7f6a2c10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 --2- 172.21.15.47:0/4087387900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2c103530 0x7f6a2c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 --2- 172.21.15.47:0/4087387900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c105e60 0x7f6a2c108250 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 -- 172.21.15.47:0/4087387900 >> 172.21.15.47:0/4087387900 conn(0x7f6a2c0fcf40 msgr2=0x7f6a2c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 -- 172.21.15.47:0/4087387900 shutdown_connections 2024-04-25T20:59:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 -- 172.21.15.47:0/4087387900 wait complete. 2024-04-25T20:59:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 Processor -- start 2024-04-25T20:59:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.886+0000 7f6a3404b700 1 -- start start 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a3404b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2c103530 0x7f6a2c196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a3404b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a2c105e60 0x7f6a2c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a3404b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c108790 0x7f6a2c19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a3404b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c10c360 con 0x7f6a2c108790 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a3404b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c10c1e0 con 0x7f6a2c105e60 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a3404b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c10c4e0 con 0x7f6a2c103530 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c108790 0x7f6a2c19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a31de7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2c103530 0x7f6a2c196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c108790 0x7f6a2c19df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37960/0 (socket says 172.21.15.47:37960) 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 -- 172.21.15.47:0/2347541009 learned_addr learned my addr 172.21.15.47:0/2347541009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a315e6700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a2c105e60 0x7f6a2c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 -- 172.21.15.47:0/2347541009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2c103530 msgr2=0x7f6a2c196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2c103530 0x7f6a2c196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 -- 172.21.15.47:0/2347541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a2c105e60 msgr2=0x7f6a2c196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a2c105e60 0x7f6a2c196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 -- 172.21.15.47:0/2347541009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c19e5f0 con 0x7f6a2c108790 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a315e6700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a2c105e60 0x7f6a2c196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a325e8700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c108790 0x7f6a2c19df50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e190 tx=0x7f6a2400e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a22ffd700 1 -- 172.21.15.47:0/2347541009 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24016440 con 0x7f6a2c108790 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c19e8e0 con 0x7f6a2c108790 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a22ffd700 1 -- 172.21.15.47:0/2347541009 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2401b070 con 0x7f6a2c108790 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.887+0000 7f6a22ffd700 1 -- 172.21.15.47:0/2347541009 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24015940 con 0x7f6a2c108790 2024-04-25T20:59:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.888+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c19ee20 con 0x7f6a2c108790 2024-04-25T20:59:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.889+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c069730 con 0x7f6a2c108790 2024-04-25T20:59:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.891+0000 7f6a22ffd700 1 -- 172.21.15.47:0/2347541009 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6a24016c40 con 0x7f6a2c108790 2024-04-25T20:59:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.891+0000 7f6a22ffd700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a180a3c40 0x7f6a180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.891+0000 7f6a31de7700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a180a3c40 0x7f6a180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.891+0000 7f6a22ffd700 1 -- 172.21.15.47:0/2347541009 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f6a240d4670 con 0x7f6a2c108790 2024-04-25T20:59:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.891+0000 7f6a31de7700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a180a3c40 0x7f6a180a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a28005950 tx=0x7f6a2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:15.895+0000 7f6a22ffd700 1 -- 172.21.15.47:0/2347541009 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a240a0fe0 con 0x7f6a2c108790 2024-04-25T20:59:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:15 smithi186 ceph-mon[32529]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:16.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:15 smithi092 ceph-mon[32524]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:16.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.124+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f6a2c062e80 con 0x7f6a180a3c40 2024-04-25T20:59:16.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.125+0000 7f6a22ffd700 1 -- 172.21.15.47:0/2347541009 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f6a2c062e80 con 0x7f6a180a3c40 2024-04-25T20:59:16.126 INFO:teuthology.orchestra.run.smithi047.stdout:Backend: cephadm 2024-04-25T20:59:16.126 INFO:teuthology.orchestra.run.smithi047.stdout:Available: Yes 2024-04-25T20:59:16.126 INFO:teuthology.orchestra.run.smithi047.stdout:Paused: No 2024-04-25T20:59:16.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a180a3c40 msgr2=0x7f6a180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:16.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a180a3c40 0x7f6a180a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a28005950 tx=0x7f6a2800b410 comp rx=0 tx=0).stop 2024-04-25T20:59:16.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c108790 msgr2=0x7f6a2c19df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:16.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c108790 0x7f6a2c19df50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e190 tx=0x7f6a2400e4a0 comp rx=0 tx=0).stop 2024-04-25T20:59:16.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 shutdown_connections 2024-04-25T20:59:16.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f6a180a3c40 0x7f6a180a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:16.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c108790 0x7f6a2c19df50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:16.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2c103530 0x7f6a2c196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:16.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 --2- 172.21.15.47:0/2347541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a2c105e60 0x7f6a2c196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:16.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 >> 172.21.15.47:0/2347541009 conn(0x7f6a2c0fcf40 msgr2=0x7f6a2c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:16.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 shutdown_connections 2024-04-25T20:59:16.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:16.127+0000 7f6a3404b700 1 -- 172.21.15.47:0/2347541009 wait complete. 2024-04-25T20:59:16.770 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T20:59:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:17 smithi047 ceph-mon[29195]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:17.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:17 smithi047 ceph-mon[29195]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:17.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:17 smithi186 ceph-mon[32529]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:17.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:17 smithi186 ceph-mon[32529]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:17 smithi092 ceph-mon[32524]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:18.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:17 smithi092 ceph-mon[32524]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1844312198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd581019a0 msgr2=0x7fdd58101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1844312198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd581019a0 0x7fdd58101dc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800b0d0 tx=0x7fdd4800b3e0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1844312198 shutdown_connections 2024-04-25T20:59:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1844312198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd581019a0 0x7fdd58101dc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1844312198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd58103540 0x7fdd58109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1844312198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd58102ba0 0x7fdd58103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1844312198 >> 172.21.15.47:0/1844312198 conn(0x7fdd580fcf40 msgr2=0x7fdd580ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1844312198 shutdown_connections 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.663+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1844312198 wait complete. 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd5ca63700 1 Processor -- start 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd5ca63700 1 -- start start 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd5ca63700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd581019a0 0x7fdd58198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd5ca63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58102ba0 0x7fdd58199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd5ca63700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd58103540 0x7fdd581a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd5ca63700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5810d490 con 0x7fdd58102ba0 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd5ca63700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5810d310 con 0x7fdd581019a0 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd5ca63700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5810d610 con 0x7fdd58103540 2024-04-25T20:59:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd55d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58102ba0 0x7fdd58199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd55d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58102ba0 0x7fdd58199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37984/0 (socket says 172.21.15.47:37984) 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.664+0000 7fdd55d9b700 1 -- 172.21.15.47:0/1985681466 learned_addr learned my addr 172.21.15.47:0/1985681466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd5659c700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd581019a0 0x7fdd58198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd56d9d700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd58103540 0x7fdd581a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd55d9b700 1 -- 172.21.15.47:0/1985681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd58103540 msgr2=0x7fdd581a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd55d9b700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd58103540 0x7fdd581a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd55d9b700 1 -- 172.21.15.47:0/1985681466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd581019a0 msgr2=0x7fdd58198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd55d9b700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd581019a0 0x7fdd58198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd55d9b700 1 -- 172.21.15.47:0/1985681466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd581a0930 con 0x7fdd58102ba0 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd56d9d700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd58103540 0x7fdd581a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd55d9b700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58102ba0 0x7fdd58199220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b530 tx=0x7fdd4000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd4f7fe700 1 -- 172.21.15.47:0/1985681466 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40015020 con 0x7fdd58102ba0 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd4f7fe700 1 -- 172.21.15.47:0/1985681466 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd40011ce0 con 0x7fdd58102ba0 2024-04-25T20:59:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd4f7fe700 1 -- 172.21.15.47:0/1985681466 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40010d20 con 0x7fdd58102ba0 2024-04-25T20:59:18.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd581a0c20 con 0x7fdd58102ba0 2024-04-25T20:59:18.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.665+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd581a1160 con 0x7fdd58102ba0 2024-04-25T20:59:18.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.666+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd581a1390 con 0x7fdd58102ba0 2024-04-25T20:59:18.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.667+0000 7fdd4f7fe700 1 -- 172.21.15.47:0/1985681466 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd40018020 con 0x7fdd58102ba0 2024-04-25T20:59:18.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.668+0000 7fdd4f7fe700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd440a3cc0 0x7fdd440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:18.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.668+0000 7fdd5659c700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd440a3cc0 0x7fdd440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:18.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.668+0000 7fdd4f7fe700 1 -- 172.21.15.47:0/1985681466 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fdd400d0160 con 0x7fdd58102ba0 2024-04-25T20:59:18.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.669+0000 7fdd5659c700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd440a3cc0 0x7fdd440a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800d010 tx=0x7fdd4800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.672+0000 7fdd4f7fe700 1 -- 172.21.15.47:0/1985681466 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd4001f020 con 0x7fdd58102ba0 2024-04-25T20:59:18.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.892+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdd5806ae30 con 0x7fdd440a3cc0 2024-04-25T20:59:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.906+0000 7fdd4f7fe700 1 -- 172.21.15.47:0/1985681466 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fdd5806ae30 con 0x7fdd440a3cc0 2024-04-25T20:59:18.907 INFO:teuthology.orchestra.run.smithi047.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T20:59:18.907 INFO:teuthology.orchestra.run.smithi047.stdout:mds.a smithi047 running (2m) 23s ago 2m 16.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6b2424b7f203 2024-04-25T20:59:18.907 INFO:teuthology.orchestra.run.smithi047.stdout:mds.b smithi092 running (2m) 25s ago 2m 27.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2e73444aa077 2024-04-25T20:59:18.907 INFO:teuthology.orchestra.run.smithi047.stdout:mds.c smithi186 running (2m) 25s ago 2m 18.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3e8dbccef698 2024-04-25T20:59:18.907 INFO:teuthology.orchestra.run.smithi047.stdout:mds.d smithi047 running (3m) 23s ago 3m 14.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 279f0681eb37 2024-04-25T20:59:18.907 INFO:teuthology.orchestra.run.smithi047.stdout:mds.e smithi092 running (3m) 25s ago 3m 22.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 edd06b8f0236 2024-04-25T20:59:18.907 INFO:teuthology.orchestra.run.smithi047.stdout:mds.f smithi186 running (3m) 25s ago 2m 19.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 dd9898b1495e 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mds.g smithi047 running (2m) 23s ago 2m 14.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2f3d7d4e3667 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mds.h smithi092 running (2m) 25s ago 2m 21.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c2973cdf983e 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mds.i smithi186 running (2m) 25s ago 2m 22.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 655565dc0158 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mds.j smithi047 running (3m) 23s ago 3m 14.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 594afccd3cb2 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mds.k smithi092 running (3m) 25s ago 3m 14.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 76869c1f7439 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mds.l smithi186 running (3m) 25s ago 3m 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 de88cee1ac37 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.x smithi047 *:9283,8765 running (10m) 23s ago 10m 516M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9dba1d645dcf 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.y smithi092 *:8443,8765 running (8m) 25s ago 8m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fa68289a9a10 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.z smithi186 *:8443,8765 running (8m) 25s ago 8m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0bac3ee556bb 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mon.a smithi047 running (10m) 23s ago 10m 51.3M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 0248222cfd5e 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mon.b smithi092 running (8m) 25s ago 8m 39.6M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 68d25731cb93 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:mon.c smithi186 running (8m) 25s ago 8m 39.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 7220ee8fe3f6 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 smithi047 running (8m) 23s ago 8m 50.8M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 34a1b5c44213 2024-04-25T20:59:18.908 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 smithi092 running (7m) 25s ago 7m 51.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 eae8bb4dc5f7 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 smithi186 running (7m) 25s ago 7m 50.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 80c9c018eb41 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 smithi047 running (6m) 23s ago 6m 51.0M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 b317f6556a86 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 smithi092 running (6m) 25s ago 6m 46.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 1cee575c5839 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 smithi186 running (6m) 25s ago 6m 57.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 287478ade75a 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 smithi047 running (5m) 23s ago 5m 54.5M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 b796941df15a 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 smithi092 running (5m) 25s ago 5m 50.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 747946a74be5 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 smithi186 running (4m) 25s ago 4m 47.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 71c931e8d298 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 smithi047 running (4m) 23s ago 4m 48.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 e83588f1eae4 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 smithi092 running (4m) 25s ago 4m 55.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 57ad398166e5 2024-04-25T20:59:18.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 smithi186 running (3m) 25s ago 3m 46.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 4265d58000db 2024-04-25T20:59:18.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.907+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd440a3cc0 msgr2=0x7fdd440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:18.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.907+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd440a3cc0 0x7fdd440a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800d010 tx=0x7fdd4800b040 comp rx=0 tx=0).stop 2024-04-25T20:59:18.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58102ba0 msgr2=0x7fdd58199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:18.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58102ba0 0x7fdd58199220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b530 tx=0x7fdd4000b8f0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 shutdown_connections 2024-04-25T20:59:18.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fdd440a3cc0 0x7fdd440a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd58103540 0x7fdd581a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd581019a0 0x7fdd58198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 --2- 172.21.15.47:0/1985681466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58102ba0 0x7fdd58199220 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:18.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 >> 172.21.15.47:0/1985681466 conn(0x7fdd580fcf40 msgr2=0x7fdd5806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:18.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 shutdown_connections 2024-04-25T20:59:18.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:18.908+0000 7fdd5ca63700 1 -- 172.21.15.47:0/1985681466 wait complete. 2024-04-25T20:59:19.618 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T20:59:19.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:19 smithi047 ceph-mon[29195]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:19.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:19 smithi186 ceph-mon[32529]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:20.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:19 smithi092 ceph-mon[32524]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:20.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:20 smithi047 ceph-mon[29195]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:20.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:20 smithi186 ceph-mon[32529]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:21.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:20 smithi092 ceph-mon[32524]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:21.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 -- 172.21.15.47:0/1422896195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 msgr2=0x7fb088102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:21.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 --2- 172.21.15.47:0/1422896195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 0x7fb088102ee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb0780099c0 tx=0x7fb078009cd0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 -- 172.21.15.47:0/1422896195 shutdown_connections 2024-04-25T20:59:21.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 --2- 172.21.15.47:0/1422896195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 0x7fb088102ee0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 --2- 172.21.15.47:0/1422896195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb088104660 0x7fb08810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 --2- 172.21.15.47:0/1422896195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb088103cc0 0x7fb088104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 -- 172.21.15.47:0/1422896195 >> 172.21.15.47:0/1422896195 conn(0x7fb0880fe060 msgr2=0x7fb0881004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 -- 172.21.15.47:0/1422896195 shutdown_connections 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.534+0000 7fb08cc75700 1 -- 172.21.15.47:0/1422896195 wait complete. 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.535+0000 7fb08cc75700 1 Processor -- start 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.535+0000 7fb08cc75700 1 -- start start 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.535+0000 7fb08cc75700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 0x7fb088198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.535+0000 7fb08cc75700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb088103cc0 0x7fb0881990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.535+0000 7fb08cc75700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb088104660 0x7fb0881a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.535+0000 7fb08cc75700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb08810e550 con 0x7fb088102ac0 2024-04-25T20:59:21.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.535+0000 7fb08cc75700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb08810e3d0 con 0x7fb088103cc0 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.535+0000 7fb08cc75700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb08810e6d0 con 0x7fb088104660 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 0x7fb088198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 0x7fb088198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38006/0 (socket says 172.21.15.47:38006) 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 -- 172.21.15.47:0/3449386730 learned_addr learned my addr 172.21.15.47:0/3449386730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb086d9d700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb088104660 0x7fb0881a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb085d9b700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb088103cc0 0x7fb0881990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 -- 172.21.15.47:0/3449386730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb088104660 msgr2=0x7fb0881a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb088104660 0x7fb0881a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 -- 172.21.15.47:0/3449386730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb088103cc0 msgr2=0x7fb0881990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb088103cc0 0x7fb0881990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 -- 172.21.15.47:0/3449386730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0881a07f0 con 0x7fb088102ac0 2024-04-25T20:59:21.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08659c700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 0x7fb088198b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb078009f50 tx=0x7fb0780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:21.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb07f7fe700 1 -- 172.21.15.47:0/3449386730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078027070 con 0x7fb088102ac0 2024-04-25T20:59:21.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0881a0a80 con 0x7fb088102ac0 2024-04-25T20:59:21.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb07f7fe700 1 -- 172.21.15.47:0/3449386730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb078004020 con 0x7fb088102ac0 2024-04-25T20:59:21.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb07f7fe700 1 -- 172.21.15.47:0/3449386730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078022ca0 con 0x7fb088102ac0 2024-04-25T20:59:21.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.536+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0881a0fb0 con 0x7fb088102ac0 2024-04-25T20:59:21.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.537+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb088069730 con 0x7fb088102ac0 2024-04-25T20:59:21.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.539+0000 7fb07f7fe700 1 -- 172.21.15.47:0/3449386730 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb07802a070 con 0x7fb088102ac0 2024-04-25T20:59:21.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.539+0000 7fb07f7fe700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb0740a3c40 0x7fb0740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:21.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.539+0000 7fb085d9b700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb0740a3c40 0x7fb0740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:21.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.539+0000 7fb07f7fe700 1 -- 172.21.15.47:0/3449386730 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fb0780d1750 con 0x7fb088102ac0 2024-04-25T20:59:21.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.540+0000 7fb085d9b700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb0740a3c40 0x7fb0740a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb070004630 tx=0x7fb07000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:21.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.543+0000 7fb07f7fe700 1 -- 172.21.15.47:0/3449386730 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb078031070 con 0x7fb088102ac0 2024-04-25T20:59:21.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.760+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb088062e80 con 0x7fb0740a3c40 2024-04-25T20:59:21.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.767+0000 7fb07f7fe700 1 -- 172.21.15.47:0/3449386730 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb088062e80 con 0x7fb0740a3c40 2024-04-25T20:59:21.768 INFO:teuthology.orchestra.run.smithi047.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T20:59:21.768 INFO:teuthology.orchestra.run.smithi047.stdout:mds.all 12/12 27s ago 3m smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-25T20:59:21.768 INFO:teuthology.orchestra.run.smithi047.stdout:mgr 3/3 27s ago 8m smithi047=x;smithi092=y;smithi186=z;count:3 2024-04-25T20:59:21.768 INFO:teuthology.orchestra.run.smithi047.stdout:mon 3/3 27s ago 8m smithi047:172.21.15.47=a;smithi092:172.21.15.92=b;smithi186:172.21.15.186=c;count:3 2024-04-25T20:59:21.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd 12 27s ago - 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb0740a3c40 msgr2=0x7fb0740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb0740a3c40 0x7fb0740a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb070004630 tx=0x7fb07000b3c0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 msgr2=0x7fb088198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 0x7fb088198b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb078009f50 tx=0x7fb0780047b0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 shutdown_connections 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fb0740a3c40 0x7fb0740a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088102ac0 0x7fb088198b90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb088104660 0x7fb0881a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 --2- 172.21.15.47:0/3449386730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb088103cc0 0x7fb0881990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:21.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 >> 172.21.15.47:0/3449386730 conn(0x7fb0880fe060 msgr2=0x7fb088107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:21.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 shutdown_connections 2024-04-25T20:59:21.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:21.769+0000 7fb08cc75700 1 -- 172.21.15.47:0/3449386730 wait complete. 2024-04-25T20:59:21.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:21 smithi047 ceph-mon[29195]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:21.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:21 smithi186 ceph-mon[32529]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:22.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:21 smithi092 ceph-mon[32524]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:22.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T20:59:22.702 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:22.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:22 smithi047 ceph-mon[29195]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:22.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:22 smithi186 ceph-mon[32529]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:23.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:22 smithi092 ceph-mon[32524]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:23.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:23 smithi047 ceph-mon[29195]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:23.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:23 smithi186 ceph-mon[32529]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:24.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:23 smithi092 ceph-mon[32524]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 -- 172.21.15.47:0/2574876992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2841019a0 msgr2=0x7ff284101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 --2- 172.21.15.47:0/2574876992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2841019a0 0x7ff284101dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2740099c0 tx=0x7ff274009cd0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 -- 172.21.15.47:0/2574876992 shutdown_connections 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 --2- 172.21.15.47:0/2574876992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2841019a0 0x7ff284101dc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 --2- 172.21.15.47:0/2574876992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff284103540 0x7ff284109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 --2- 172.21.15.47:0/2574876992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff284102ba0 0x7ff284103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 -- 172.21.15.47:0/2574876992 >> 172.21.15.47:0/2574876992 conn(0x7ff2840fcf40 msgr2=0x7ff2840ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 -- 172.21.15.47:0/2574876992 shutdown_connections 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.335+0000 7ff28c53f700 1 -- 172.21.15.47:0/2574876992 wait complete. 2024-04-25T20:59:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28c53f700 1 Processor -- start 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28c53f700 1 -- start start 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28c53f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2841019a0 0x7ff2841058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28c53f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284102ba0 0x7ff284103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28c53f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff284103540 0x7ff2841044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28c53f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff28410d410 con 0x7ff284102ba0 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28c53f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff28410d290 con 0x7ff284103540 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28c53f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff28410d590 con 0x7ff2841019a0 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff289ada700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284102ba0 0x7ff284103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff289ada700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284102ba0 0x7ff284103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37506/0 (socket says 172.21.15.47:37506) 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff289ada700 1 -- 172.21.15.47:0/385946510 learned_addr learned my addr 172.21.15.47:0/385946510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.336+0000 7ff28a2db700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2841019a0 0x7ff2841058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff289ada700 1 -- 172.21.15.47:0/385946510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2841019a0 msgr2=0x7ff2841058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff289ada700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2841019a0 0x7ff2841058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff289ada700 1 -- 172.21.15.47:0/385946510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff284103540 msgr2=0x7ff2841044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff28aadc700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff284103540 0x7ff2841044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff289ada700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff284103540 0x7ff2841044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff289ada700 1 -- 172.21.15.47:0/385946510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff284104d90 con 0x7ff284102ba0 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff289ada700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284102ba0 0x7ff284103f80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c920 tx=0x7ff28000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff27b7fe700 1 -- 172.21.15.47:0/385946510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280007b00 con 0x7ff284102ba0 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2841a2db0 con 0x7ff284102ba0 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff27b7fe700 1 -- 172.21.15.47:0/385946510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280004ce0 con 0x7ff284102ba0 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff27b7fe700 1 -- 172.21.15.47:0/385946510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280005550 con 0x7ff284102ba0 2024-04-25T20:59:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.337+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2841a31b0 con 0x7ff284102ba0 2024-04-25T20:59:24.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.338+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff284069760 con 0x7ff284102ba0 2024-04-25T20:59:24.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.339+0000 7ff27b7fe700 1 -- 172.21.15.47:0/385946510 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff280007ca0 con 0x7ff284102ba0 2024-04-25T20:59:24.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.340+0000 7ff27b7fe700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff2700a3c40 0x7ff2700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:24.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.340+0000 7ff28a2db700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff2700a3c40 0x7ff2700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:24.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.340+0000 7ff27b7fe700 1 -- 172.21.15.47:0/385946510 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7ff2800d03e0 con 0x7ff284102ba0 2024-04-25T20:59:24.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.340+0000 7ff28a2db700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff2700a3c40 0x7ff2700a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff2740036a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.344+0000 7ff27b7fe700 1 -- 172.21.15.47:0/385946510 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff284069760 con 0x7ff284102ba0 2024-04-25T20:59:24.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.558+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7ff28406ae60 con 0x7ff2700a3c40 2024-04-25T20:59:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.560+0000 7ff27b7fe700 1 -- 172.21.15.47:0/385946510 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7ff28406ae60 con 0x7ff2700a3c40 2024-04-25T20:59:24.560 INFO:teuthology.orchestra.run.smithi047.stdout:HOST ADDR LABELS STATUS 2024-04-25T20:59:24.561 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 172.21.15.47 2024-04-25T20:59:24.561 INFO:teuthology.orchestra.run.smithi047.stdout:smithi092 172.21.15.92 2024-04-25T20:59:24.561 INFO:teuthology.orchestra.run.smithi047.stdout:smithi186 172.21.15.186 2024-04-25T20:59:24.561 INFO:teuthology.orchestra.run.smithi047.stdout:3 hosts in cluster 2024-04-25T20:59:24.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff2700a3c40 msgr2=0x7ff2700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff2700a3c40 0x7ff2700a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff2740036a0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284102ba0 msgr2=0x7ff284103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284102ba0 0x7ff284103f80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c920 tx=0x7ff28000cc30 comp rx=0 tx=0).stop 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 shutdown_connections 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7ff2700a3c40 0x7ff2700a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff284103540 0x7ff2841044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2841019a0 0x7ff2841058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 --2- 172.21.15.47:0/385946510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284102ba0 0x7ff284103f80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 >> 172.21.15.47:0/385946510 conn(0x7ff2840fcf40 msgr2=0x7ff28406a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 shutdown_connections 2024-04-25T20:59:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:24.562+0000 7ff28c53f700 1 -- 172.21.15.47:0/385946510 wait complete. 2024-04-25T20:59:25.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T20:59:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:25.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:25 smithi047 ceph-mon[29195]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:25.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:25 smithi047 ceph-mon[29195]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:25.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:25 smithi186 ceph-mon[32529]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:25.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:25 smithi186 ceph-mon[32529]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:25 smithi092 ceph-mon[32524]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:26.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:25 smithi092 ceph-mon[32524]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.057+0000 7f7acb1c3700 1 -- 172.21.15.47:0/1718370950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4103ca0 msgr2=0x7f7ac4104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.057+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/1718370950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4103ca0 0x7f7ac4104100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00099c0 tx=0x7f7ab0009cd0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.057+0000 7f7acb1c3700 1 -- 172.21.15.47:0/1718370950 shutdown_connections 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.057+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/1718370950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ac4104640 0x7f7ac410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.057+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/1718370950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4102aa0 0x7f7ac4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/1718370950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4103ca0 0x7f7ac4104100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 -- 172.21.15.47:0/1718370950 >> 172.21.15.47:0/1718370950 conn(0x7f7ac40fe040 msgr2=0x7f7ac4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 -- 172.21.15.47:0/1718370950 shutdown_connections 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 -- 172.21.15.47:0/1718370950 wait complete. 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 Processor -- start 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 -- start start 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4102aa0 0x7f7ac4198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ac4103ca0 0x7f7ac4199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4104640 0x7f7ac41a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac410e510 con 0x7f7ac4104640 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac410e390 con 0x7f7ac4102aa0 2024-04-25T20:59:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.058+0000 7f7acb1c3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac410e690 con 0x7f7ac4103ca0 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4104640 0x7f7ac41a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4104640 0x7f7ac41a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37534/0 (socket says 172.21.15.47:37534) 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 -- 172.21.15.47:0/3785547023 learned_addr learned my addr 172.21.15.47:0/3785547023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac8f5f700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4102aa0 0x7f7ac4198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac3fff700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ac4103ca0 0x7f7ac4199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 -- 172.21.15.47:0/3785547023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ac4103ca0 msgr2=0x7f7ac4199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ac4103ca0 0x7f7ac4199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 -- 172.21.15.47:0/3785547023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4102aa0 msgr2=0x7f7ac4198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4102aa0 0x7f7ac4198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 -- 172.21.15.47:0/3785547023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac41a07b0 con 0x7f7ac4104640 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac3fff700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ac4103ca0 0x7f7ac4199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac9760700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4104640 0x7f7ac41a0110 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7abc007da0 tx=0x7f7abc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac1ffb700 1 -- 172.21.15.47:0/3785547023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc016020 con 0x7f7ac4104640 2024-04-25T20:59:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac1ffb700 1 -- 172.21.15.47:0/3785547023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc00ad00 con 0x7f7ac4104640 2024-04-25T20:59:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac41a0aa0 con 0x7f7ac4104640 2024-04-25T20:59:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.059+0000 7f7ac1ffb700 1 -- 172.21.15.47:0/3785547023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0115e0 con 0x7f7ac4104640 2024-04-25T20:59:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.060+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac41a0f90 con 0x7f7ac4104640 2024-04-25T20:59:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.061+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac4069730 con 0x7f7ac4104640 2024-04-25T20:59:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.061+0000 7f7ac1ffb700 1 -- 172.21.15.47:0/3785547023 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7abc019020 con 0x7f7ac4104640 2024-04-25T20:59:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.062+0000 7f7ac1ffb700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7ab40a3c40 0x7f7ab40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.062+0000 7f7ac1ffb700 1 -- 172.21.15.47:0/3785547023 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f7abc0d0470 con 0x7f7ac4104640 2024-04-25T20:59:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.062+0000 7f7ac8f5f700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7ab40a3c40 0x7f7ab40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.063+0000 7f7ac8f5f700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7ab40a3c40 0x7f7ab40a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4103b00 tx=0x7f7ab800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.067+0000 7f7ac1ffb700 1 -- 172.21.15.47:0/3785547023 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7abc020020 con 0x7f7ac4104640 2024-04-25T20:59:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.286+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7ac4062e80 con 0x7f7ab40a3c40 2024-04-25T20:59:27.289 INFO:teuthology.orchestra.run.smithi047.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T20:59:27.289 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130022480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-04-25T20:59:27.289 INFO:teuthology.orchestra.run.smithi047.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T20:59:27.289 INFO:teuthology.orchestra.run.smithi047.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T20:59:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.288+0000 7f7ac1ffb700 1 -- 172.21.15.47:0/3785547023 <== mgr.14152 v2:172.21.15.47:6800/253800519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7ac4062e80 con 0x7f7ab40a3c40 2024-04-25T20:59:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.290+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7ab40a3c40 msgr2=0x7f7ab40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.290+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7ab40a3c40 0x7f7ab40a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4103b00 tx=0x7f7ab800b410 comp rx=0 tx=0).stop 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4104640 msgr2=0x7f7ac41a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4104640 0x7f7ac41a0110 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7abc007da0 tx=0x7f7abc00a340 comp rx=0 tx=0).stop 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 shutdown_connections 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f7ab40a3c40 0x7f7ab40a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac4104640 0x7f7ac41a0110 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4102aa0 0x7f7ac4198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 --2- 172.21.15.47:0/3785547023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ac4103ca0 0x7f7ac4199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 >> 172.21.15.47:0/3785547023 conn(0x7f7ac40fe040 msgr2=0x7f7ac4107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 shutdown_connections 2024-04-25T20:59:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:27.291+0000 7f7acb1c3700 1 -- 172.21.15.47:0/3785547023 wait complete. 2024-04-25T20:59:27.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:27 smithi047 ceph-mon[29195]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:27.907 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T20:59:27.916 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T20:59:27.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T20:59:27.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:27 smithi186 ceph-mon[32529]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:28.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:27 smithi092 ceph-mon[32524]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:28.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:28 smithi047 ceph-mon[29195]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:28 smithi186 ceph-mon[32529]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:29.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:28 smithi092 ceph-mon[32524]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:29.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.769+0000 7fda713c4700 1 -- 172.21.15.47:0/2982377545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c104730 msgr2=0x7fda6c10afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:29.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.769+0000 7fda713c4700 1 --2- 172.21.15.47:0/2982377545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c104730 0x7fda6c10afe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fda6400b0a0 tx=0x7fda6400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:29.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.769+0000 7fda713c4700 1 -- 172.21.15.47:0/2982377545 shutdown_connections 2024-04-25T20:59:29.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.769+0000 7fda713c4700 1 --2- 172.21.15.47:0/2982377545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c104730 0x7fda6c10afe0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:29.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.769+0000 7fda713c4700 1 --2- 172.21.15.47:0/2982377545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c100db0 0x7fda6c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:29.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.769+0000 7fda713c4700 1 --2- 172.21.15.47:0/2982377545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6c103d90 0x7fda6c1041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:29.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.769+0000 7fda713c4700 1 -- 172.21.15.47:0/2982377545 >> 172.21.15.47:0/2982377545 conn(0x7fda6c0fc710 msgr2=0x7fda6c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:29.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.769+0000 7fda713c4700 1 -- 172.21.15.47:0/2982377545 shutdown_connections 2024-04-25T20:59:29.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 -- 172.21.15.47:0/2982377545 wait complete. 2024-04-25T20:59:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 Processor -- start 2024-04-25T20:59:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 -- start start 2024-04-25T20:59:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c100db0 0x7fda6c1036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6c103d90 0x7fda6c101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:29.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c104730 0x7fda6c102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10e560 con 0x7fda6c100db0 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10e3e0 con 0x7fda6c103d90 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.770+0000 7fda713c4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10e6e0 con 0x7fda6c104730 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c100db0 0x7fda6c1036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c100db0 0x7fda6c1036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37556/0 (socket says 172.21.15.47:37556) 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 -- 172.21.15.47:0/801299109 learned_addr learned my addr 172.21.15.47:0/801299109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6b7fe700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c104730 0x7fda6c102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6a7fc700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6c103d90 0x7fda6c101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 -- 172.21.15.47:0/801299109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c104730 msgr2=0x7fda6c102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c104730 0x7fda6c102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 -- 172.21.15.47:0/801299109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6c103d90 msgr2=0x7fda6c101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6c103d90 0x7fda6c101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 -- 172.21.15.47:0/801299109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c102b30 con 0x7fda6c100db0 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda6affd700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c100db0 0x7fda6c1036f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00d980 tx=0x7fda5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda53fff700 1 -- 172.21.15.47:0/801299109 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c017030 con 0x7fda6c100db0 2024-04-25T20:59:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c1a2d00 con 0x7fda6c100db0 2024-04-25T20:59:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda53fff700 1 -- 172.21.15.47:0/801299109 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda5c0044d0 con 0x7fda6c100db0 2024-04-25T20:59:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda53fff700 1 -- 172.21.15.47:0/801299109 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c003970 con 0x7fda6c100db0 2024-04-25T20:59:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.771+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c1a3190 con 0x7fda6c100db0 2024-04-25T20:59:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.772+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c02df80 con 0x7fda6c100db0 2024-04-25T20:59:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.774+0000 7fda53fff700 1 -- 172.21.15.47:0/801299109 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fda5c01a070 con 0x7fda6c100db0 2024-04-25T20:59:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.775+0000 7fda53fff700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fda540a3c40 0x7fda540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.775+0000 7fda6a7fc700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fda540a3c40 0x7fda540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.775+0000 7fda53fff700 1 -- 172.21.15.47:0/801299109 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fda5c0d0000 con 0x7fda6c100db0 2024-04-25T20:59:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.775+0000 7fda6a7fc700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fda540a3c40 0x7fda540a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda60005950 tx=0x7fda6000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:29.778+0000 7fda53fff700 1 -- 172.21.15.47:0/801299109 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda5c00ee20 con 0x7fda6c100db0 2024-04-25T20:59:29.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:29 smithi047 ceph-mon[29195]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:29.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:29 smithi186 ceph-mon[32529]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:30.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.031+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fda6c0636f0 con 0x7fda6c100db0 2024-04-25T20:59:30.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.031+0000 7fda53fff700 1 -- 172.21.15.47:0/801299109 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3038 (secure 0 0 0) 0x7fda5c09c9d0 con 0x7fda6c100db0 2024-04-25T20:59:30.032 INFO:teuthology.orchestra.run.smithi047.stdout:e25 2024-04-25T20:59:30.032 INFO:teuthology.orchestra.run.smithi047.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:59:30.032 INFO:teuthology.orchestra.run.smithi047.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:legacy client fscid: 1 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:Filesystem 'cephfs' (1) 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:fs_name cephfs 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 25 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-04-25T20:56:53.085968+0000 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-04-25T20:57:11.813557+0000 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:tableserver 0 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:root 0 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:session_timeout 300 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:session_autoclose 300 2024-04-25T20:59:30.033 INFO:teuthology.orchestra.run.smithi047.stdout:max_file_size 1099511627776 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:max_xattr_size 65536 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:required_client_features {} 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure 0 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure_osd_epoch 0 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:max_mds 3 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:in 0,1,2 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:up {0=24488,1=24467,2=24452} 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:failed 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:damaged 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:stopped 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:data_pools [3] 2024-04-25T20:59:30.034 INFO:teuthology.orchestra.run.smithi047.stdout:metadata_pool 2 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:inline_data disabled 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:balancer 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:bal_rank_mask -1 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:standby_count_wanted 0 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:qdb_cluster leader: 24488 members: 24452,24488,24467 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b{0:24488} state up:active seq 5 addr [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.i{0:24448} state up:standby-replay seq 1 addr [v2:172.21.15.186:6838/60292465,v1:172.21.15.186:6839/60292465] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.h{1:24467} state up:active seq 12 addr [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c{1:24436} state up:standby-replay seq 1 addr [v2:172.21.15.186:6836/4095064414,v1:172.21.15.186:6837/4095064414] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.e{2:24452} state up:active seq 14 addr [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.f{2:24430} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/3860377657,v1:172.21.15.186:6835/3860377657] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:30.035 INFO:teuthology.orchestra.run.smithi047.stdout:Standby daemons: 2024-04-25T20:59:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:30.036 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.47:6834/911765730,v1:172.21.15.47:6835/911765730] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.036 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.47:6836/12948729,v1:172.21.15.47:6837/12948729] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.036 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.g{-1:14586} state up:standby seq 1 addr [v2:172.21.15.47:6838/2858318196,v1:172.21.15.47:6839/2858318196] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.036 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a{-1:14592} state up:standby seq 1 addr [v2:172.21.15.47:6840/622157648,v1:172.21.15.47:6841/622157648] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.036 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.l{-1:24400} state up:standby seq 1 addr [v2:172.21.15.186:6832/3007060622,v1:172.21.15.186:6833/3007060622] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.036 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.92:6832/1380167956,v1:172.21.15.92:6833/1380167956] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:59:30.036 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:29 smithi092 ceph-mon[32524]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fda540a3c40 msgr2=0x7fda540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fda540a3c40 0x7fda540a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda60005950 tx=0x7fda6000b410 comp rx=0 tx=0).stop 2024-04-25T20:59:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c100db0 msgr2=0x7fda6c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c100db0 0x7fda6c1036f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00d980 tx=0x7fda5c00dc90 comp rx=0 tx=0).stop 2024-04-25T20:59:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 shutdown_connections 2024-04-25T20:59:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fda540a3c40 0x7fda540a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c100db0 0x7fda6c1036f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c104730 0x7fda6c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 --2- 172.21.15.47:0/801299109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6c103d90 0x7fda6c101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.033+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 >> 172.21.15.47:0/801299109 conn(0x7fda6c0fc710 msgr2=0x7fda6c107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.034+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 shutdown_connections 2024-04-25T20:59:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:30.034+0000 7fda713c4700 1 -- 172.21.15.47:0/801299109 wait complete. 2024-04-25T20:59:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2024-04-25T20:59:30.700 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T20:59:30.949 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:30.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:30 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/801299109' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T20:59:31.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:30 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/801299109' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T20:59:31.071 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:30 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/801299109' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T20:59:31.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:31 smithi047 ceph-mon[29195]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:31.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:31 smithi186 ceph-mon[32529]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:32.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:31 smithi092 ceph-mon[32524]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:32.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 -- 172.21.15.47:0/4144698901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8103ca0 msgr2=0x7fa1d8104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:32.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 --2- 172.21.15.47:0/4144698901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8103ca0 0x7fa1d8104100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0099c0 tx=0x7fa1cc009cd0 comp rx=0 tx=0).stop 2024-04-25T20:59:32.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 -- 172.21.15.47:0/4144698901 shutdown_connections 2024-04-25T20:59:32.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 --2- 172.21.15.47:0/4144698901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d8104640 0x7fa1d810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:32.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 --2- 172.21.15.47:0/4144698901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1d8102aa0 0x7fa1d8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:32.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 --2- 172.21.15.47:0/4144698901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8103ca0 0x7fa1d8104100 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:32.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 -- 172.21.15.47:0/4144698901 >> 172.21.15.47:0/4144698901 conn(0x7fa1d80fe040 msgr2=0x7fa1d8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 -- 172.21.15.47:0/4144698901 shutdown_connections 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.832+0000 7fa1de05a700 1 -- 172.21.15.47:0/4144698901 wait complete. 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.833+0000 7fa1de05a700 1 Processor -- start 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.833+0000 7fa1de05a700 1 -- start start 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.833+0000 7fa1de05a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8102aa0 0x7fa1d8198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.833+0000 7fa1de05a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d8103ca0 0x7fa1d81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.833+0000 7fa1de05a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1d8104640 0x7fa1d81a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1de05a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d810e630 con 0x7fa1d8102aa0 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1de05a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d810e4b0 con 0x7fa1d8103ca0 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1de05a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d810e7b0 con 0x7fa1d8104640 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8102aa0 0x7fa1d8198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8102aa0 0x7fa1d8198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37574/0 (socket says 172.21.15.47:37574) 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 -- 172.21.15.47:0/2581850598 learned_addr learned my addr 172.21.15.47:0/2581850598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:32.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d7fff700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1d8104640 0x7fa1d81a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d6ffd700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d8103ca0 0x7fa1d81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 -- 172.21.15.47:0/2581850598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1d8104640 msgr2=0x7fa1d81a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1d8104640 0x7fa1d81a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 -- 172.21.15.47:0/2581850598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d8103ca0 msgr2=0x7fa1d81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d8103ca0 0x7fa1d81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 -- 172.21.15.47:0/2581850598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d81a09a0 con 0x7fa1d8102aa0 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d77fe700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8102aa0 0x7fa1d8198d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800b530 tx=0x7fa1c800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d4ff9700 1 -- 172.21.15.47:0/2581850598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8015020 con 0x7fa1d8102aa0 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d81a0c90 con 0x7fa1d8102aa0 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d4ff9700 1 -- 172.21.15.47:0/2581850598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c8011ce0 con 0x7fa1d8102aa0 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.834+0000 7fa1d4ff9700 1 -- 172.21.15.47:0/2581850598 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8010d60 con 0x7fa1d8102aa0 2024-04-25T20:59:32.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.835+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d81a1170 con 0x7fa1d8102aa0 2024-04-25T20:59:32.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.836+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d802df80 con 0x7fa1d8102aa0 2024-04-25T20:59:32.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.837+0000 7fa1d4ff9700 1 -- 172.21.15.47:0/2581850598 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa1c8018020 con 0x7fa1d8102aa0 2024-04-25T20:59:32.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.837+0000 7fa1d4ff9700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1c00a3c40 0x7fa1c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:32.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.837+0000 7fa1d6ffd700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1c00a3c40 0x7fa1c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:32.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.837+0000 7fa1d4ff9700 1 -- 172.21.15.47:0/2581850598 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fa1c80cfbc0 con 0x7fa1d8102aa0 2024-04-25T20:59:32.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.838+0000 7fa1d6ffd700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1c00a3c40 0x7fa1c00a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc003890 tx=0x7fa1cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:32.842+0000 7fa1d4ff9700 1 -- 172.21.15.47:0/2581850598 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1c801f020 con 0x7fa1d8102aa0 2024-04-25T20:59:33.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.049+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa1d8063650 con 0x7fa1d8102aa0 2024-04-25T20:59:33.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.050+0000 7fa1d4ff9700 1 -- 172.21.15.47:0/2581850598 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5291 (secure 0 0 0) 0x7fa1c809c530 con 0x7fa1d8102aa0 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 91 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-04-25T20:48:51.766698+0000 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-04-25T20:57:47.520070+0000 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:crush_version 26 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:full_ratio 0.95 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:backfillfull_ratio 0.9 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:nearfull_ratio 0.85 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:require_min_compat_client luminous 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:min_compat_client luminous 2024-04-25T20:59:33.051 INFO:teuthology.orchestra.run.smithi047.stdout:require_osd_release squid 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:stretch_mode_enabled false 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:max_osd 12 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6802/3737942264,v1:172.21.15.47:6803/3737942264] [v2:172.21.15.47:6804/3737942264,v1:172.21.15.47:6805/3737942264] exists,up 4b824d92-28a7-4a8e-b2fe-1d6512481dea 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3070209756,v1:172.21.15.92:6801/3070209756] [v2:172.21.15.92:6802/3070209756,v1:172.21.15.92:6803/3070209756] exists,up cef4a330-707b-41e0-bf53-12a1a03da237 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/651738121,v1:172.21.15.186:6801/651738121] [v2:172.21.15.186:6802/651738121,v1:172.21.15.186:6803/651738121] exists,up 49a4bac4-8c22-47d9-b96f-56e99bd8cec0 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 up in weight 1 up_from 25 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6810/2086583036,v1:172.21.15.47:6811/2086583036] [v2:172.21.15.47:6812/2086583036,v1:172.21.15.47:6813/2086583036] exists,up 0f303ea7-0cef-436c-82ed-e3a7f1fde8ab 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 up in weight 1 up_from 33 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/477256695,v1:172.21.15.92:6809/477256695] [v2:172.21.15.92:6810/477256695,v1:172.21.15.92:6811/477256695] exists,up 8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 up in weight 1 up_from 39 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/2592795959,v1:172.21.15.186:6809/2592795959] [v2:172.21.15.186:6810/2592795959,v1:172.21.15.186:6811/2592795959] exists,up 21521287-3d9f-4f8d-aa39-4a2d79d2cf34 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6818/2069873922,v1:172.21.15.47:6819/2069873922] [v2:172.21.15.47:6820/2069873922,v1:172.21.15.47:6821/2069873922] exists,up a7726429-12e9-4dd8-9bae-8aa979f0c447 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 up in weight 1 up_from 52 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/1310010410,v1:172.21.15.92:6817/1310010410] [v2:172.21.15.92:6818/1310010410,v1:172.21.15.92:6819/1310010410] exists,up e9f86109-f4e5-4770-88b9-6454fa1927bd 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 up in weight 1 up_from 60 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/773299526,v1:172.21.15.186:6817/773299526] [v2:172.21.15.186:6818/773299526,v1:172.21.15.186:6819/773299526] exists,up df80bc74-f3eb-43e1-ac44-7061cdddf622 2024-04-25T20:59:33.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 up in weight 1 up_from 68 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6826/3118226797,v1:172.21.15.47:6827/3118226797] [v2:172.21.15.47:6828/3118226797,v1:172.21.15.47:6829/3118226797] exists,up b313eacf-dc05-4b28-a5d4-4d04e7b1bda9 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 up in weight 1 up_from 74 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/3185760943,v1:172.21.15.92:6825/3185760943] [v2:172.21.15.92:6826/3185760943,v1:172.21.15.92:6827/3185760943] exists,up 7a31b12e-d889-4a51-9060-b79255c81626 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 up in weight 1 up_from 81 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/936644453,v1:172.21.15.186:6825/936644453] [v2:172.21.15.186:6826/936644453,v1:172.21.15.186:6827/936644453] exists,up 92dc80e5-665e-412c-9596-b4ea0f65e561 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.6 [8,11] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.1d [2,4] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.0 [5,9] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.2 [3,9] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.6 [4,6] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.e [4,7] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.19 [3,8] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.1d [4,8,5,9] 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1508622913 expires 2024-04-26T20:49:45.811145+0000 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/2212759557 expires 2024-04-26T20:49:45.811145+0000 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/1508582803 expires 2024-04-26T20:49:18.960239+0000 2024-04-25T20:59:33.053 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/1508582803 expires 2024-04-26T20:49:18.960239+0000 2024-04-25T20:59:33.054 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1792012036 expires 2024-04-26T20:49:18.960239+0000 2024-04-25T20:59:33.054 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/2195749439 expires 2024-04-26T20:49:45.811145+0000 2024-04-25T20:59:33.054 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/2351249200 expires 2024-04-26T20:49:45.811145+0000 2024-04-25T20:59:33.054 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/2078121135 expires 2024-04-26T20:49:18.960239+0000 2024-04-25T20:59:33.054 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/795796209 expires 2024-04-26T20:49:18.960239+0000 2024-04-25T20:59:33.054 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/2212759557 expires 2024-04-26T20:49:45.811145+0000 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1c00a3c40 msgr2=0x7fa1c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1c00a3c40 0x7fa1c00a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc003890 tx=0x7fa1cc005fb0 comp rx=0 tx=0).stop 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8102aa0 msgr2=0x7fa1d8198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8102aa0 0x7fa1d8198d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800b530 tx=0x7fa1c800b840 comp rx=0 tx=0).stop 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 shutdown_connections 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fa1c00a3c40 0x7fa1c00a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d8102aa0 0x7fa1d8198d90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1d8104640 0x7fa1d81a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 --2- 172.21.15.47:0/2581850598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d8103ca0 0x7fa1d81992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 >> 172.21.15.47:0/2581850598 conn(0x7fa1d80fe040 msgr2=0x7fa1d8107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 shutdown_connections 2024-04-25T20:59:33.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:33.052+0000 7fa1de05a700 1 -- 172.21.15.47:0/2581850598 wait complete. 2024-04-25T20:59:33.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:33 smithi047 ceph-mon[29195]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:33.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:33 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2581850598' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T20:59:33.771 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T20:59:33.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:59:33.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:33 smithi186 ceph-mon[32529]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:33.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:33 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2581850598' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T20:59:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:33 smithi092 ceph-mon[32524]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:34.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:33 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2581850598' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T20:59:34.040 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 -- 172.21.15.47:0/761793939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 msgr2=0x7f96e810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 --2- 172.21.15.47:0/761793939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 0x7f96e810f930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f96e000b0a0 tx=0x7f96e000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 -- 172.21.15.47:0/761793939 shutdown_connections 2024-04-25T20:59:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 --2- 172.21.15.47:0/761793939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 0x7f96e810f930 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 --2- 172.21.15.47:0/761793939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8108a60 0x7f96e8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 --2- 172.21.15.47:0/761793939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e806b1f0 0x7f96e806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 -- 172.21.15.47:0/761793939 >> 172.21.15.47:0/761793939 conn(0x7f96e80789e0 msgr2=0x7f96e8078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 -- 172.21.15.47:0/761793939 shutdown_connections 2024-04-25T20:59:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 -- 172.21.15.47:0/761793939 wait complete. 2024-04-25T20:59:35.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.754+0000 7f96ed802700 1 Processor -- start 2024-04-25T20:59:35.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96ed802700 1 -- start start 2024-04-25T20:59:35.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96ed802700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e806b1f0 0x7f96e81019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:35.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96ed802700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 0x7f96e8101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:35.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96ed802700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8108a60 0x7f96e8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96ed802700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8112ca0 con 0x7f96e806bbb0 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96ed802700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8112b20 con 0x7f96e8108a60 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96ed802700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8112e20 con 0x7f96e806b1f0 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 0x7f96e8101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 0x7f96e8101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52746/0 (socket says 172.21.15.47:52746) 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e67fc700 1 -- 172.21.15.47:0/313197568 learned_addr learned my addr 172.21.15.47:0/313197568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e77fe700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8108a60 0x7f96e8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e6ffd700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e806b1f0 0x7f96e81019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e67fc700 1 -- 172.21.15.47:0/313197568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e806b1f0 msgr2=0x7f96e81019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e67fc700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e806b1f0 0x7f96e81019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e67fc700 1 -- 172.21.15.47:0/313197568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8108a60 msgr2=0x7f96e8102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e67fc700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8108a60 0x7f96e8102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e67fc700 1 -- 172.21.15.47:0/313197568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e81a75f0 con 0x7f96e806bbb0 2024-04-25T20:59:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.755+0000 7f96e6ffd700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e806b1f0 0x7f96e81019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.756+0000 7f96e67fc700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 0x7f96e8101ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00b530 tx=0x7f96dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.756+0000 7f96ec800700 1 -- 172.21.15.47:0/313197568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc015020 con 0x7f96e806bbb0 2024-04-25T20:59:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.756+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e81a78e0 con 0x7f96e806bbb0 2024-04-25T20:59:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.756+0000 7f96ec800700 1 -- 172.21.15.47:0/313197568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96dc011ce0 con 0x7f96e806bbb0 2024-04-25T20:59:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.756+0000 7f96ec800700 1 -- 172.21.15.47:0/313197568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc010e40 con 0x7f96e806bbb0 2024-04-25T20:59:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.756+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81a7e20 con 0x7f96e806bbb0 2024-04-25T20:59:35.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.757+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e8069730 con 0x7f96e806bbb0 2024-04-25T20:59:35.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.758+0000 7f96ec800700 1 -- 172.21.15.47:0/313197568 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f96dc018020 con 0x7f96e806bbb0 2024-04-25T20:59:35.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.759+0000 7f96ec800700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f96d00a3c40 0x7f96d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:35.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.759+0000 7f96e6ffd700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f96d00a3c40 0x7f96d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:35.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.759+0000 7f96ec800700 1 -- 172.21.15.47:0/313197568 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f96dc0cfcc0 con 0x7f96e806bbb0 2024-04-25T20:59:35.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.759+0000 7f96e6ffd700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f96d00a3c40 0x7f96d00a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96d8005fd0 tx=0x7f96d8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:35.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:35.763+0000 7f96ec800700 1 -- 172.21.15.47:0/313197568 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96dc01f020 con 0x7f96e806bbb0 2024-04-25T20:59:35.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:35 smithi047 ceph-mon[29195]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:35.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:35 smithi186 ceph-mon[32529]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.020+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96e8102e90 con 0x7f96e806bbb0 2024-04-25T20:59:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.021+0000 7f96ec800700 1 -- 172.21.15.47:0/313197568 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11853 (secure 0 0 0) 0x7f96dc09c690 con 0x7f96e806bbb0 2024-04-25T20:59:36.022 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:36.022 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:57:11.813557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24488,24467]},"id":1}]} 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f96d00a3c40 msgr2=0x7f96d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f96d00a3c40 0x7f96d00a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96d8005fd0 tx=0x7f96d8005e20 comp rx=0 tx=0).stop 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 msgr2=0x7f96e8101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 0x7f96e8101ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00b530 tx=0x7f96dc00b840 comp rx=0 tx=0).stop 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 shutdown_connections 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f96d00a3c40 0x7f96d00a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8108a60 0x7f96e8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e806b1f0 0x7f96e81019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 --2- 172.21.15.47:0/313197568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e806bbb0 0x7f96e8101ef0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 >> 172.21.15.47:0/313197568 conn(0x7f96e80789e0 msgr2=0x7f96e80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:36.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 shutdown_connections 2024-04-25T20:59:36.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:36.023+0000 7f96ed802700 1 -- 172.21.15.47:0/313197568 wait complete. 2024-04-25T20:59:36.025 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2024-04-25T20:59:36.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:35 smithi092 ceph-mon[32524]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:36.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:59:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:36 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/313197568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:37.018 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:37.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:36 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/313197568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:37.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:36 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/313197568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:37.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:37 smithi186 ceph-mon[32529]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:38.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:37 smithi092 ceph-mon[32524]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:38.085 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:37 smithi047 ceph-mon[29195]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 -- 172.21.15.47:0/455276868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c108a30 msgr2=0x7f9a6c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 --2- 172.21.15.47:0/455276868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c108a30 0x7f9a6c108e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009990 tx=0x7f9a54009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 -- 172.21.15.47:0/455276868 shutdown_connections 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 --2- 172.21.15.47:0/455276868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c108a30 0x7f9a6c108e10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 --2- 172.21.15.47:0/455276868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a6c06b0f0 0x7f9a6c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 --2- 172.21.15.47:0/455276868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c06a750 0x7f9a6c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 -- 172.21.15.47:0/455276868 >> 172.21.15.47:0/455276868 conn(0x7f9a6c0788f0 msgr2=0x7f9a6c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 -- 172.21.15.47:0/455276868 shutdown_connections 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.611+0000 7f9a716d4700 1 -- 172.21.15.47:0/455276868 wait complete. 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a716d4700 1 Processor -- start 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a716d4700 1 -- start start 2024-04-25T20:59:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a716d4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c06a750 0x7f9a6c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:38.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a716d4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a6c06b0f0 0x7f9a6c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:38.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a716d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c108a30 0x7f9a6c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a716d4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c112bb0 con 0x7f9a6c06a750 2024-04-25T20:59:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a716d4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c112a30 con 0x7f9a6c108a30 2024-04-25T20:59:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a716d4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c112d30 con 0x7f9a6c06b0f0 2024-04-25T20:59:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a6affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c06a750 0x7f9a6c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a6affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c06a750 0x7f9a6c19b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52762/0 (socket says 172.21.15.47:52762) 2024-04-25T20:59:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.612+0000 7f9a6affd700 1 -- 172.21.15.47:0/1920607276 learned_addr learned my addr 172.21.15.47:0/1920607276 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a6a7fc700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a6c06b0f0 0x7f9a6c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a6b7fe700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c108a30 0x7f9a6c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a6affd700 1 -- 172.21.15.47:0/1920607276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a6c06b0f0 msgr2=0x7f9a6c19b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a6affd700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a6c06b0f0 0x7f9a6c19b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a6affd700 1 -- 172.21.15.47:0/1920607276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c108a30 msgr2=0x7f9a6c19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a6affd700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c108a30 0x7f9a6c19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a6affd700 1 -- 172.21.15.47:0/1920607276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c1a0270 con 0x7f9a6c06a750 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a6affd700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c06a750 0x7f9a6c19b260 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9a540057f0 tx=0x7f9a54003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a63fff700 1 -- 172.21.15.47:0/1920607276 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54027070 con 0x7f9a6c06a750 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6c1a0500 con 0x7f9a6c06a750 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a63fff700 1 -- 172.21.15.47:0/1920607276 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a540041c0 con 0x7f9a6c06a750 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a63fff700 1 -- 172.21.15.47:0/1920607276 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54022e60 con 0x7f9a6c06a750 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.613+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a6c1a7de0 con 0x7f9a6c06a750 2024-04-25T20:59:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.614+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a6c069730 con 0x7f9a6c06a750 2024-04-25T20:59:38.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.616+0000 7f9a63fff700 1 -- 172.21.15.47:0/1920607276 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9a5402a070 con 0x7f9a6c06a750 2024-04-25T20:59:38.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.616+0000 7f9a63fff700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9a580a3c40 0x7f9a580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:38.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.616+0000 7f9a6a7fc700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9a580a3c40 0x7f9a580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:38.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.616+0000 7f9a63fff700 1 -- 172.21.15.47:0/1920607276 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f9a540d1730 con 0x7f9a6c06a750 2024-04-25T20:59:38.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.617+0000 7f9a6a7fc700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9a580a3c40 0x7f9a580a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c005fd0 tx=0x7f9a5c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:38.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.624+0000 7f9a63fff700 1 -- 172.21.15.47:0/1920607276 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a54013070 con 0x7f9a6c06a750 2024-04-25T20:59:38.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.899+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a6c1a7fb0 con 0x7f9a6c06a750 2024-04-25T20:59:38.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.900+0000 7f9a63fff700 1 -- 172.21.15.47:0/1920607276 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11853 (secure 0 0 0) 0x7f9a5409e190 con 0x7f9a6c06a750 2024-04-25T20:59:38.900 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:38.901 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:57:11.813557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24488,24467]},"id":1}]} 2024-04-25T20:59:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9a580a3c40 msgr2=0x7f9a580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9a580a3c40 0x7f9a580a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c005fd0 tx=0x7f9a5c00a560 comp rx=0 tx=0).stop 2024-04-25T20:59:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c06a750 msgr2=0x7f9a6c19b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c06a750 0x7f9a6c19b260 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9a540057f0 tx=0x7f9a54003730 comp rx=0 tx=0).stop 2024-04-25T20:59:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 shutdown_connections 2024-04-25T20:59:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f9a580a3c40 0x7f9a580a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c06a750 0x7f9a6c19b260 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c108a30 0x7f9a6c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 --2- 172.21.15.47:0/1920607276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a6c06b0f0 0x7f9a6c19b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 >> 172.21.15.47:0/1920607276 conn(0x7f9a6c0788f0 msgr2=0x7f9a6c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 shutdown_connections 2024-04-25T20:59:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:38.902+0000 7f9a716d4700 1 -- 172.21.15.47:0/1920607276 wait complete. 2024-04-25T20:59:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2024-04-25T20:59:39.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:59:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:39.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:39 smithi047 ceph-mon[29195]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:39.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:39 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1920607276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:39.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:39 smithi186 ceph-mon[32529]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:39.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:39 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1920607276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:40.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:39 smithi092 ceph-mon[32524]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:40.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:39 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1920607276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 -- 172.21.15.47:0/698741357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 msgr2=0x7f2394116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 --2- 172.21.15.47:0/698741357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 0x7f2394116030 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f238400b0a0 tx=0x7f238400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 -- 172.21.15.47:0/698741357 shutdown_connections 2024-04-25T20:59:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 --2- 172.21.15.47:0/698741357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 0x7f2394116030 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 --2- 172.21.15.47:0/698741357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2394074c70 0x7f2394075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 --2- 172.21.15.47:0/698741357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2394075590 0x7f2394113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 -- 172.21.15.47:0/698741357 >> 172.21.15.47:0/698741357 conn(0x7f2394079bb0 msgr2=0x7f2394079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 -- 172.21.15.47:0/698741357 shutdown_connections 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.512+0000 7f239be4e700 1 -- 172.21.15.47:0/698741357 wait complete. 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239be4e700 1 Processor -- start 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239be4e700 1 -- start start 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239be4e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2394074c70 0x7f23941a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239be4e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2394075590 0x7f23941a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239be4e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 0x7f239419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239be4e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f23941192a0 con 0x7f2394113c40 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239be4e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2394119120 con 0x7f2394074c70 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239be4e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2394119420 con 0x7f2394075590 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239a3eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 0x7f239419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239a3eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 0x7f239419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52768/0 (socket says 172.21.15.47:52768) 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239a3eb700 1 -- 172.21.15.47:0/926369501 learned_addr learned my addr 172.21.15.47:0/926369501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f2399bea700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2394074c70 0x7f23941a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f23993e9700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2394075590 0x7f23941a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239a3eb700 1 -- 172.21.15.47:0/926369501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2394075590 msgr2=0x7f23941a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239a3eb700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2394075590 0x7f23941a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239a3eb700 1 -- 172.21.15.47:0/926369501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2394074c70 msgr2=0x7f23941a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239a3eb700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2394074c70 0x7f23941a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.513+0000 7f239a3eb700 1 -- 172.21.15.47:0/926369501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239419ff40 con 0x7f2394113c40 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.514+0000 7f23993e9700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2394075590 0x7f23941a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.514+0000 7f239a3eb700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 0x7f239419f660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f23840062a0 tx=0x7f2384004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.514+0000 7f2392ffd700 1 -- 172.21.15.47:0/926369501 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384021070 con 0x7f2394113c40 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.514+0000 7f2392ffd700 1 -- 172.21.15.47:0/926369501 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2384007e30 con 0x7f2394113c40 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.514+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23941a01d0 con 0x7f2394113c40 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.514+0000 7f2392ffd700 1 -- 172.21.15.47:0/926369501 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23840047a0 con 0x7f2394113c40 2024-04-25T20:59:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.514+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941ac510 con 0x7f2394113c40 2024-04-25T20:59:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.515+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2394076360 con 0x7f2394113c40 2024-04-25T20:59:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.516+0000 7f2392ffd700 1 -- 172.21.15.47:0/926369501 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2384007980 con 0x7f2394113c40 2024-04-25T20:59:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.517+0000 7f2392ffd700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f237c0a3cc0 0x7f237c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.517+0000 7f2399bea700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f237c0a3cc0 0x7f237c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.517+0000 7f2392ffd700 1 -- 172.21.15.47:0/926369501 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f23840d1420 con 0x7f2394113c40 2024-04-25T20:59:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.517+0000 7f2399bea700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f237c0a3cc0 0x7f237c0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f238c009a10 tx=0x7f238c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.521+0000 7f2392ffd700 1 -- 172.21.15.47:0/926369501 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f238409de80 con 0x7f2394113c40 2024-04-25T20:59:41.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.730+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23941a0b90 con 0x7f2394113c40 2024-04-25T20:59:41.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.732+0000 7f2392ffd700 1 -- 172.21.15.47:0/926369501 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22599 (secure 0 0 0) 0x7f238409d780 con 0x7f2394113c40 2024-04-25T20:59:41.733 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T20:59:41.733 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:57:47.520070+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f237c0a3cc0 msgr2=0x7f237c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f237c0a3cc0 0x7f237c0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f238c009a10 tx=0x7f238c009190 comp rx=0 tx=0).stop 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 msgr2=0x7f239419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 0x7f239419f660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f23840062a0 tx=0x7f2384004ce0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 shutdown_connections 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f237c0a3cc0 0x7f237c0a6180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2394113c40 0x7f239419f660 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2394074c70 0x7f23941a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 --2- 172.21.15.47:0/926369501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2394075590 0x7f23941a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 >> 172.21.15.47:0/926369501 conn(0x7f2394079bb0 msgr2=0x7f2394112a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:41.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 shutdown_connections 2024-04-25T20:59:41.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:41.734+0000 7f239be4e700 1 -- 172.21.15.47:0/926369501 wait complete. 2024-04-25T20:59:41.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:41 smithi047 ceph-mon[29195]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:41.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:41 smithi186 ceph-mon[32529]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:42.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:41 smithi092 ceph-mon[32524]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:42.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T20:56:53.085968+0000', 'modified': '2024-04-25T20:57:11.813557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24467, 'mds_2': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/3860377657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 3860377657}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 3860377657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/4095064414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 4095064414}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 4095064414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6839/60292465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 60292465}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 60292465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.92:6835/2097108178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 2097108178}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 2097108178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/1984144918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1984144918}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1984144918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/941998037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 941998037}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 941998037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24452, 24488, 24467]} 2024-04-25T20:59:42.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T20:59:42.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.e status 2024-04-25T20:59:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:42.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:42 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/926369501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:59:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:42 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/926369501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:59:43.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:42 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/926369501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:59:43.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:43 smithi047 ceph-mon[29195]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:43.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:43 smithi186 ceph-mon[32529]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:44.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:43 smithi092 ceph-mon[32524]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 -- 172.21.15.47:0/3738769628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec101940 msgr2=0x7f22ec101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 --2- 172.21.15.47:0/3738769628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec101940 0x7f22ec101dc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009990 tx=0x7f22d8009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 -- 172.21.15.47:0/3738769628 shutdown_connections 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 --2- 172.21.15.47:0/3738769628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22ec102300 0x7f22ec10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 --2- 172.21.15.47:0/3738769628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22ec107990 0x7f22ec107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 --2- 172.21.15.47:0/3738769628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec101940 0x7f22ec101dc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 -- 172.21.15.47:0/3738769628 >> 172.21.15.47:0/3738769628 conn(0x7f22ec0788f0 msgr2=0x7f22ec078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 -- 172.21.15.47:0/3738769628 shutdown_connections 2024-04-25T20:59:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.318+0000 7f22f2b26700 1 -- 172.21.15.47:0/3738769628 wait complete. 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f2b26700 1 Processor -- start 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f2b26700 1 -- start start 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f2b26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22ec101940 0x7f22ec1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f2b26700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22ec102300 0x7f22ec103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f2b26700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107990 0x7f22ec103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f2b26700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec111ae0 con 0x7f22ec107990 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f2b26700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec111960 con 0x7f22ec101940 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f2b26700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec111c60 con 0x7f22ec102300 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f10c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107990 0x7f22ec103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f10c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107990 0x7f22ec103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39610/0 (socket says 172.21.15.47:39610) 2024-04-25T20:59:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f10c3700 1 -- 172.21.15.47:0/2910963981 learned_addr learned my addr 172.21.15.47:0/2910963981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22f08c2700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22ec101940 0x7f22ec1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.319+0000 7f22ebfff700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22ec102300 0x7f22ec103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22f10c3700 1 -- 172.21.15.47:0/2910963981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22ec102300 msgr2=0x7f22ec103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22f10c3700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22ec102300 0x7f22ec103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22f10c3700 1 -- 172.21.15.47:0/2910963981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22ec101940 msgr2=0x7f22ec1034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22f10c3700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22ec101940 0x7f22ec1034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22f10c3700 1 -- 172.21.15.47:0/2910963981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec1a75a0 con 0x7f22ec107990 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22f10c3700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107990 0x7f22ec103f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22e4007ee0 tx=0x7f22e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22e9ffb700 1 -- 172.21.15.47:0/2910963981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4004bb0 con 0x7f22ec107990 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22e9ffb700 1 -- 172.21.15.47:0/2910963981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e4004d50 con 0x7f22ec107990 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22f2b26700 1 -- 172.21.15.47:0/2910963981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec1a7890 con 0x7f22ec107990 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22e9ffb700 1 -- 172.21.15.47:0/2910963981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4005920 con 0x7f22ec107990 2024-04-25T20:59:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.320+0000 7f22f2b26700 1 -- 172.21.15.47:0/2910963981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec1a7da0 con 0x7f22ec107990 2024-04-25T20:59:44.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.322+0000 7f22e9ffb700 1 -- 172.21.15.47:0/2910963981 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f22e401a020 con 0x7f22ec107990 2024-04-25T20:59:44.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.323+0000 7f22e9ffb700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f22dc0a3cc0 0x7f22dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.323+0000 7f22f08c2700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f22dc0a3cc0 0x7f22dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.323+0000 7f22e9ffb700 1 -- 172.21.15.47:0/2910963981 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f22e40cfa30 con 0x7f22ec107990 2024-04-25T20:59:44.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.323+0000 7f22f08c2700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f22dc0a3cc0 0x7f22dc0a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22e000abd0 tx=0x7f22e000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:44.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22fd3afb80 1 Processor -- start 2024-04-25T20:59:44.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22fd3afb80 1 -- start start 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907252bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907257040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f9072579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f9071a4ce0 con 0x55f9072527d0 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f90714b910 con 0x55f9072530f0 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f9071a0ca0 con 0x55f907257580 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22f08c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907252bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22f08c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907252bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39622/0 (socket says 172.21.15.47:39622) 2024-04-25T20:59:44.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22f08c2700 1 -- 172.21.15.47:0/4250603736 learned_addr learned my addr 172.21.15.47:0/4250603736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22f08c2700 1 -- 172.21.15.47:0/4250603736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 msgr2=0x55f9072579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22f08c2700 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f9072579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22f08c2700 1 -- 172.21.15.47:0/4250603736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 msgr2=0x55f907257040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22ebfff700 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907257040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22f08c2700 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907257040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.324+0000 7f22f08c2700 1 -- 172.21.15.47:0/4250603736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9070f2880 con 0x55f9072527d0 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.325+0000 7f22ebfff700 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907257040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.325+0000 7f22f08c2700 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907252bb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22e0004530 tx=0x7f22e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266a5cf38851ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.325+0000 7f22d6ffd700 1 -- 172.21.15.47:0/4250603736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0016070 con 0x55f9072527d0 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.325+0000 7f22d6ffd700 1 -- 172.21.15.47:0/4250603736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e0014070 con 0x55f9072527d0 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.325+0000 7f22d6ffd700 1 -- 172.21.15.47:0/4250603736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0010e40 con 0x55f9072527d0 2024-04-25T20:59:44.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 -- 172.21.15.47:0/4250603736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 msgr2=0x55f907252bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907252bb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22e0004530 tx=0x7f22e0012000 comp rx=0 tx=0).stop 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 -- 172.21.15.47:0/4250603736 shutdown_connections 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907252bb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f9072579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/4250603736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907257040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 -- 172.21.15.47:0/4250603736 >> 172.21.15.47:0/4250603736 conn(0x55f90717f640 msgr2=0x55f90724d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 -- 172.21.15.47:0/4250603736 shutdown_connections 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 -- 172.21.15.47:0/4250603736 wait complete. 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 -- start start 2024-04-25T20:59:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.326+0000 7f22fd3afb80 1 Processor -- start 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072527d0 0x55f90725ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072530f0 0x55f90725bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f90725c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f90725f610 con 0x55f9072530f0 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f907143b80 con 0x55f9072527d0 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f90725f790 con 0x55f907257580 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22fd3afb80 10 client.0 _ll_register_callbacks cb 0x55f90713d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072530f0 0x55f90725bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072530f0 0x55f90725bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39624/0 (socket says 172.21.15.47:39624) 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 -- 172.21.15.47:0/3095105192 learned_addr learned my addr 172.21.15.47:0/3095105192 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22f08c2700 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072527d0 0x55f90725ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22f10c3700 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f90725c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 -- 172.21.15.47:0/3095105192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 msgr2=0x55f90725c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f90725c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 -- 172.21.15.47:0/3095105192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072527d0 msgr2=0x55f90725ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072527d0 0x55f90725ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 -- 172.21.15.47:0/3095105192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f90715c220 con 0x55f9072530f0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22ebfff700 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072530f0 0x55f90725bdd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009990 tx=0x7f22d800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea779a9cfadf9e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.327+0000 7f22cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22cbfff700 1 -- 172.21.15.47:0/3095105192 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8003c70 con 0x55f9072530f0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22cbfff700 1 -- 172.21.15.47:0/3095105192 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f22d8005070 con 0x55f9072530f0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3095105192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e0014070 con 0x55f9072530f0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22cbfff700 1 -- 172.21.15.47:0/3095105192 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22d8003680 con 0x55f9072530f0 2024-04-25T20:59:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22fd3afb80 10 client.15066 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3095105192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f907269ec0 con 0x55f9072530f0 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22cbfff700 1 -- 172.21.15.47:0/3095105192 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e0014070 con 0x55f9072530f0 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22cbfff700 1 -- 172.21.15.47:0/3095105192 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f22d802b020 con 0x55f9072530f0 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22fd3afb80 10 client.15066 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22fd3afb80 10 client.15066 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22fd3afb80 20 client.15066 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.328+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x55f90726c500 0x55f90726d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.329+0000 7f22fd3afb80 4 client.15066 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.329+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3095105192 --> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9071b24d0 con 0x55f90726c500 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.329+0000 7f22f08c2700 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x55f90726c500 0x55f90726d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.329+0000 7f22cbfff700 10 client.15066 ms_handle_connect on v2:172.21.15.92:6834/2097108178 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.329+0000 7f22f08c2700 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x55f90726c500 0x55f90726d970 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46eb929a8c1ebf81 server_cookie=e8173ebe7bd75ec5 in_seq=0 out_seq=0 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22cbfff700 1 -- 172.21.15.47:0/3095105192 <== mds.2 v2:172.21.15.92:6834/2097108178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9071b24d0 con 0x55f90726c500 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22cbfff700 10 client.15066 handle_command_reply: tid=0 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 client.15066 shutdown 2024-04-25T20:59:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3095105192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072530f0 msgr2=0x55f90725bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072530f0 0x55f90725bdd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009990 tx=0x7f22d800bf20 comp rx=0 tx=0).stop 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3095105192 shutdown_connections 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x55f90726c500 0x55f90726d970 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f90725c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072527d0 0x55f90725ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3095105192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072530f0 0x55f90725bdd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22cbfff700 0 client.15066 ms_handle_reset on v2:172.21.15.92:6834/2097108178 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3095105192 >> 172.21.15.47:0/3095105192 conn(0x55f90717f640 msgr2=0x55f9072589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3095105192 shutdown_connections 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.330+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3095105192 wait complete. 2024-04-25T20:59:44.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.331+0000 7f22fd3afb80 20 client.15066 trim_cache size 0 max 16384 2024-04-25T20:59:44.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22fd3afb80 1 Processor -- start 2024-04-25T20:59:44.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22fd3afb80 1 -- start start 2024-04-25T20:59:44.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f90725e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907276dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f907277310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f90726bcb0 con 0x55f9072527d0 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f90726c380 con 0x55f9072530f0 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f90726ca50 con 0x55f907257580 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22f08c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f90725e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22f08c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f90725e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39638/0 (socket says 172.21.15.47:39638) 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.351+0000 7f22f08c2700 1 -- 172.21.15.47:0/3354077865 learned_addr learned my addr 172.21.15.47:0/3354077865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22ebfff700 1 --2- 172.21.15.47:0/3354077865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907276dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22f08c2700 1 -- 172.21.15.47:0/3354077865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 msgr2=0x55f907277310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22f08c2700 1 --2- 172.21.15.47:0/3354077865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f907277310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22f08c2700 1 -- 172.21.15.47:0/3354077865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 msgr2=0x55f907276dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22f08c2700 1 --2- 172.21.15.47:0/3354077865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907276dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22f08c2700 1 -- 172.21.15.47:0/3354077865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f90726e8d0 con 0x55f9072527d0 2024-04-25T20:59:44.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22f08c2700 1 --2- 172.21.15.47:0/3354077865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f90725e8a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f22e0010010 tx=0x7f22e000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01623921a9db632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:44.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22d67fc700 1 -- 172.21.15.47:0/3354077865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0037070 con 0x55f9072527d0 2024-04-25T20:59:44.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22d67fc700 1 -- 172.21.15.47:0/3354077865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e0014030 con 0x55f9072527d0 2024-04-25T20:59:44.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.352+0000 7f22d67fc700 1 -- 172.21.15.47:0/3354077865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0016040 con 0x55f9072527d0 2024-04-25T20:59:44.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3354077865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 msgr2=0x55f90725e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3354077865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f90725e8a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f22e0010010 tx=0x7f22e000a3f0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3354077865 shutdown_connections 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3354077865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f90725e8a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3354077865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f907277310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/3354077865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907276dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3354077865 >> 172.21.15.47:0/3354077865 conn(0x55f90717f640 msgr2=0x55f90724c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3354077865 shutdown_connections 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 -- 172.21.15.47:0/3354077865 wait complete. 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 -- start start 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.353+0000 7f22fd3afb80 1 Processor -- start 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907259130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907259670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22fd3afb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f907275420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f90726d7f0 con 0x55f9072527d0 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f90726d120 con 0x55f9072530f0 2024-04-25T20:59:44.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22fd3afb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f90726dec0 con 0x55f907257580 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22fd3afb80 10 client.0 _ll_register_callbacks cb 0x55f90713d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907259130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907259130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39642/0 (socket says 172.21.15.47:39642) 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 -- 172.21.15.47:0/1682103104 learned_addr learned my addr 172.21.15.47:0/1682103104 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22ebfff700 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907259670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f10c3700 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f907275420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 -- 172.21.15.47:0/1682103104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 msgr2=0x55f907275420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f907275420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 -- 172.21.15.47:0/1682103104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 msgr2=0x55f907259670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907259670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 -- 172.21.15.47:0/1682103104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f907275960 con 0x55f9072527d0 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f10c3700 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f907275420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22f08c2700 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907259130 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22e000a480 tx=0x7f22e0010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8438a92eb6f6061c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:44.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22cbfff700 1 -- 172.21.15.47:0/1682103104 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0016070 con 0x55f9072527d0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22cbfff700 1 -- 172.21.15.47:0/1682103104 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f22e0003ca0 con 0x55f9072527d0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.354+0000 7f22fd3afb80 1 -- 172.21.15.47:0/1682103104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e0014030 con 0x55f9072527d0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22cbfff700 1 -- 172.21.15.47:0/1682103104 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22e0037070 con 0x55f9072527d0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22fd3afb80 10 client.15072 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22fd3afb80 1 -- 172.21.15.47:0/1682103104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9072d3e70 con 0x55f9072527d0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22cbfff700 1 -- 172.21.15.47:0/1682103104 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e0014030 con 0x55f9072527d0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22cbfff700 1 -- 172.21.15.47:0/1682103104 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f22e0004860 con 0x55f9072527d0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22fd3afb80 10 client.15072 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22fd3afb80 10 client.15072 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22fd3afb80 20 client.15072 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x55f9072d82e0 0x55f9072d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22fd3afb80 4 client.15072 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22fd3afb80 1 -- 172.21.15.47:0/1682103104 --> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] -- command(tid 0: {"prefix": "status"}) -- 0x55f9071ac570 con 0x55f9072d82e0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.355+0000 7f22ebfff700 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x55f9072d82e0 0x55f9072d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:44.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22ebfff700 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x55f9072d82e0 0x55f9072d6ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc4814400f364e97 server_cookie=675a5dd99e271fa1 in_seq=0 out_seq=0 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22cbfff700 10 client.15072 ms_handle_connect on v2:172.21.15.92:6834/2097108178 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22cbfff700 1 -- 172.21.15.47:0/1682103104 <== mds.2 v2:172.21.15.92:6834/2097108178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f9071ac570 con 0x55f9072d82e0 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22cbfff700 10 client.15072 handle_command_reply: tid=0 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 client.15072 shutdown 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 -- 172.21.15.47:0/1682103104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 msgr2=0x55f907259130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907259130 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22e000a480 tx=0x7f22e0010430 comp rx=0 tx=0).stop 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 -- 172.21.15.47:0/1682103104 shutdown_connections 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x55f9072d82e0 0x55f9072d6ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9072527d0 0x55f907259130 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f907257580 0x55f907275420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22cbfff700 0 client.15072 ms_handle_reset on v2:172.21.15.92:6834/2097108178 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 --2- 172.21.15.47:0/1682103104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9072530f0 0x55f907259670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 -- 172.21.15.47:0/1682103104 >> 172.21.15.47:0/1682103104 conn(0x55f90717f640 msgr2=0x55f90725bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 -- 172.21.15.47:0/1682103104 shutdown_connections 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.356+0000 7f22fd3afb80 1 -- 172.21.15.47:0/1682103104 wait complete. 2024-04-25T20:59:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22fd3afb80 20 client.15072 trim_cache size 0 max 16384 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24452, 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 157.64227823600001, 2024-04-25T20:59:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 25, 2024-04-25T20:59:44.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 91, 2024-04-25T20:59:44.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T20:59:44.363 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 207.77224624199999 2024-04-25T20:59:44.363 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T20:59:44.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 -- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f22dc0a3cc0 msgr2=0x7f22dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f22dc0a3cc0 0x7f22dc0a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22e000abd0 tx=0x7f22e000a380 comp rx=0 tx=0).stop 2024-04-25T20:59:44.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 -- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107990 msgr2=0x7f22ec103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107990 0x7f22ec103f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22e4007ee0 tx=0x7f22e400d3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 -- 172.21.15.47:0/2910963981 shutdown_connections 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f22dc0a3cc0 0x7f22dc0a6180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107990 0x7f22ec103f70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22ec101940 0x7f22ec1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 --2- 172.21.15.47:0/2910963981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22ec102300 0x7f22ec103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 -- 172.21.15.47:0/2910963981 >> 172.21.15.47:0/2910963981 conn(0x7f22ec0788f0 msgr2=0x7f22ec06aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 -- 172.21.15.47:0/2910963981 shutdown_connections 2024-04-25T20:59:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:44.357+0000 7f22f2b26700 1 -- 172.21.15.47:0/2910963981 wait complete. 2024-04-25T20:59:45.087 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T20:59:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:45.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:45 smithi047 ceph-mon[29195]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:45.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:45 smithi186 ceph-mon[32529]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:46.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:45 smithi092 ceph-mon[32524]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 -- 172.21.15.47:0/1513020490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c06b1f0 msgr2=0x7fab2c06b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 --2- 172.21.15.47:0/1513020490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c06b1f0 0x7fab2c06b670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab20009990 tx=0x7fab20009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 -- 172.21.15.47:0/1513020490 shutdown_connections 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 --2- 172.21.15.47:0/1513020490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab2c06bbb0 0x7fab2c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 --2- 172.21.15.47:0/1513020490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c108a60 0x7fab2c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 --2- 172.21.15.47:0/1513020490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c06b1f0 0x7fab2c06b670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 -- 172.21.15.47:0/1513020490 >> 172.21.15.47:0/1513020490 conn(0x7fab2c0789e0 msgr2=0x7fab2c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 -- 172.21.15.47:0/1513020490 shutdown_connections 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 -- 172.21.15.47:0/1513020490 wait complete. 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.096+0000 7fab31a8d700 1 Processor -- start 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab31a8d700 1 -- start start 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab31a8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab2c06b1f0 0x7fab2c1019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab31a8d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c06bbb0 0x7fab2c101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab31a8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c108a60 0x7fab2c102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab31a8d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c112ca0 con 0x7fab2c108a60 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab31a8d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c112b20 con 0x7fab2c06b1f0 2024-04-25T20:59:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab31a8d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c112e20 con 0x7fab2c06bbb0 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c108a60 0x7fab2c102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c108a60 0x7fab2c102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39648/0 (socket says 172.21.15.47:39648) 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 -- 172.21.15.47:0/2462230201 learned_addr learned my addr 172.21.15.47:0/2462230201 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab2c06b1f0 0x7fab2c1019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2affd700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c06bbb0 0x7fab2c101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 -- 172.21.15.47:0/2462230201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c06bbb0 msgr2=0x7fab2c101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c06bbb0 0x7fab2c101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 -- 172.21.15.47:0/2462230201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab2c06b1f0 msgr2=0x7fab2c1019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab2c06b1f0 0x7fab2c1019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 -- 172.21.15.47:0/2462230201 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c1a76d0 con 0x7fab2c108a60 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab2c06b1f0 0x7fab2c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.097+0000 7fab2bfff700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c108a60 0x7fab2c102430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b970 tx=0x7fab2400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.098+0000 7fab28ff9700 1 -- 172.21.15.47:0/2462230201 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24019070 con 0x7fab2c108a60 2024-04-25T20:59:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.098+0000 7fab28ff9700 1 -- 172.21.15.47:0/2462230201 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab24014440 con 0x7fab2c108a60 2024-04-25T20:59:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.098+0000 7fab31a8d700 1 -- 172.21.15.47:0/2462230201 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c1a79c0 con 0x7fab2c108a60 2024-04-25T20:59:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.098+0000 7fab28ff9700 1 -- 172.21.15.47:0/2462230201 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2400ce60 con 0x7fab2c108a60 2024-04-25T20:59:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.098+0000 7fab31a8d700 1 -- 172.21.15.47:0/2462230201 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c1a7f00 con 0x7fab2c108a60 2024-04-25T20:59:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.100+0000 7fab28ff9700 1 -- 172.21.15.47:0/2462230201 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab2401c070 con 0x7fab2c108a60 2024-04-25T20:59:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.100+0000 7fab28ff9700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fab140a3cc0 0x7fab140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.101+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fab140a3cc0 0x7fab140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.101+0000 7fab28ff9700 1 -- 172.21.15.47:0/2462230201 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fab240cf7c0 con 0x7fab2c108a60 2024-04-25T20:59:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.101+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fab140a3cc0 0x7fab140a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005950 tx=0x7fab1c005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.101+0000 7fab3c316b80 1 Processor -- start 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.101+0000 7fab3c316b80 1 -- start start 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce36bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce3b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce3b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab3c316b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5653ccd88ce0 con 0x5653cce367d0 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab3c316b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5653ccd2f910 con 0x5653cce370f0 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab3c316b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5653ccd84ca0 con 0x5653cce3b580 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce36bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce36bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39660/0 (socket says 172.21.15.47:39660) 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 -- 172.21.15.47:0/3161907811 learned_addr learned my addr 172.21.15.47:0/3161907811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2affd700 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce3b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2bfff700 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce3b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 -- 172.21.15.47:0/3161907811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 msgr2=0x5653cce3b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce3b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 -- 172.21.15.47:0/3161907811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 msgr2=0x5653cce3b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce3b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 -- 172.21.15.47:0/3161907811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cccd6880 con 0x5653cce367d0 2024-04-25T20:59:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce36bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00ed10 tx=0x7fab1c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bad3e96a7209a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab11ffb700 1 -- 172.21.15.47:0/3161907811 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c021070 con 0x5653cce367d0 2024-04-25T20:59:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab11ffb700 1 -- 172.21.15.47:0/3161907811 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c010cd0 con 0x5653cce367d0 2024-04-25T20:59:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.102+0000 7fab11ffb700 1 -- 172.21.15.47:0/3161907811 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00f5c0 con 0x5653cce367d0 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 -- 172.21.15.47:0/3161907811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 msgr2=0x5653cce36bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce36bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00ed10 tx=0x7fab1c011000 comp rx=0 tx=0).stop 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 -- 172.21.15.47:0/3161907811 shutdown_connections 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce36bb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce3b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 --2- 172.21.15.47:0/3161907811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce3b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 -- 172.21.15.47:0/3161907811 >> 172.21.15.47:0/3161907811 conn(0x5653ccd63640 msgr2=0x5653cce31820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 -- 172.21.15.47:0/3161907811 shutdown_connections 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 -- 172.21.15.47:0/3161907811 wait complete. 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.103+0000 7fab3c316b80 1 -- start start 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab3c316b80 1 Processor -- start 2024-04-25T20:59:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce42a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce3fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce40240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab3c316b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5653cce43610 con 0x5653cce367d0 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab3c316b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5653ccd27b80 con 0x5653cce370f0 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab3c316b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5653cce43790 con 0x5653cce3b580 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab3c316b80 10 client.0 _ll_register_callbacks cb 0x5653ccd21900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce42a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce42a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39664/0 (socket says 172.21.15.47:39664) 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2b7fe700 1 -- 172.21.15.47:0/2083034848 learned_addr learned my addr 172.21.15.47:0/2083034848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2affd700 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce3fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2b7fe700 1 -- 172.21.15.47:0/2083034848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 msgr2=0x5653cce40240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce40240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2b7fe700 1 -- 172.21.15.47:0/2083034848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 msgr2=0x5653cce3fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce3fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.104+0000 7fab2b7fe700 1 -- 172.21.15.47:0/2083034848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ccd40220 con 0x5653cce367d0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce42a00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005840 tx=0x7fab1c005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9818e13cb29c732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab0affd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab0affd700 1 -- 172.21.15.47:0/2083034848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c021030 con 0x5653cce367d0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab0affd700 1 -- 172.21.15.47:0/2083034848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fab1c010ad0 con 0x5653cce367d0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab3c316b80 1 -- 172.21.15.47:0/2083034848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1c010cd0 con 0x5653cce367d0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab0affd700 1 -- 172.21.15.47:0/2083034848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab1c004900 con 0x5653cce367d0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab3c316b80 10 client.15084 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab3c316b80 1 -- 172.21.15.47:0/2083034848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653cce4de60 con 0x5653cce367d0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab0affd700 1 -- 172.21.15.47:0/2083034848 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c010cd0 con 0x5653cce367d0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab0affd700 1 -- 172.21.15.47:0/2083034848 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fab1c03a050 con 0x5653cce367d0 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab3c316b80 10 client.15084 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab3c316b80 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:59:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab3c316b80 20 client.15084 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.105+0000 7fab3c316b80 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5653cce50210 0x5653cce51680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.106+0000 7fab3c316b80 4 client.15084 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.106+0000 7fab3c316b80 1 -- 172.21.15.47:0/2083034848 --> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ccd964d0 con 0x5653cce50210 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.106+0000 7fab2affd700 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5653cce50210 0x5653cce51680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.106+0000 7fab2affd700 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5653cce50210 0x5653cce51680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b49b7627c2696603 server_cookie=b5d2cc856d9a36c9 in_seq=0 out_seq=0 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.106+0000 7fab0affd700 10 client.15084 ms_handle_connect on v2:172.21.15.92:6836/1984144918 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab0affd700 1 -- 172.21.15.47:0/2083034848 <== mds.1 v2:172.21.15.92:6836/1984144918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653ccd964d0 con 0x5653cce50210 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab0affd700 10 client.15084 handle_command_reply: tid=0 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 client.15084 shutdown 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 -- 172.21.15.47:0/2083034848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 msgr2=0x5653cce42a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce42a00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005840 tx=0x7fab1c005870 comp rx=0 tx=0).stop 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 -- 172.21.15.47:0/2083034848 shutdown_connections 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5653cce50210 0x5653cce51680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce42a00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce40240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 --2- 172.21.15.47:0/2083034848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce3fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 -- 172.21.15.47:0/2083034848 >> 172.21.15.47:0/2083034848 conn(0x5653ccd63640 msgr2=0x5653cce3c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab0affd700 0 client.15084 ms_handle_reset on v2:172.21.15.92:6836/1984144918 2024-04-25T20:59:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 -- 172.21.15.47:0/2083034848 shutdown_connections 2024-04-25T20:59:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 1 -- 172.21.15.47:0/2083034848 wait complete. 2024-04-25T20:59:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.107+0000 7fab3c316b80 20 client.15084 trim_cache size 0 max 16384 2024-04-25T20:59:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab3c316b80 1 Processor -- start 2024-04-25T20:59:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab3c316b80 1 -- start start 2024-04-25T20:59:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce367d0 0x5653cce312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce3b580 0x5653cce38f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab3c316b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5653cce4f9c0 con 0x5653cce3b580 2024-04-25T20:59:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab3c316b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5653cce50090 con 0x5653cce370f0 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab3c316b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5653cce50760 con 0x5653cce367d0 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab2bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce3b580 0x5653cce38f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab2bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce3b580 0x5653cce38f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39672/0 (socket says 172.21.15.47:39672) 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.128+0000 7fab2bfff700 1 -- 172.21.15.47:0/882034393 learned_addr learned my addr 172.21.15.47:0/882034393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce367d0 0x5653cce312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2affd700 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2bfff700 1 -- 172.21.15.47:0/882034393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce367d0 msgr2=0x5653cce312b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2bfff700 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce367d0 0x5653cce312b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2bfff700 1 -- 172.21.15.47:0/882034393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 msgr2=0x5653cce317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2bfff700 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2bfff700 1 -- 172.21.15.47:0/882034393 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cce31d30 con 0x5653cce3b580 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce367d0 0x5653cce312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab2bfff700 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce3b580 0x5653cce38f40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b940 tx=0x7fab240bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95c8b254ff6e29b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab117fa700 1 -- 172.21.15.47:0/882034393 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2409fe40 con 0x5653cce3b580 2024-04-25T20:59:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab117fa700 1 -- 172.21.15.47:0/882034393 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab240a4700 con 0x5653cce3b580 2024-04-25T20:59:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.129+0000 7fab117fa700 1 -- 172.21.15.47:0/882034393 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240a4270 con 0x5653cce3b580 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 -- 172.21.15.47:0/882034393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce3b580 msgr2=0x5653cce38f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce3b580 0x5653cce38f40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b940 tx=0x7fab240bc040 comp rx=0 tx=0).stop 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 -- 172.21.15.47:0/882034393 shutdown_connections 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce3b580 0x5653cce38f40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce367d0 0x5653cce312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 --2- 172.21.15.47:0/882034393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 -- 172.21.15.47:0/882034393 >> 172.21.15.47:0/882034393 conn(0x5653ccd63640 msgr2=0x5653cce30200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 -- 172.21.15.47:0/882034393 shutdown_connections 2024-04-25T20:59:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 -- 172.21.15.47:0/882034393 wait complete. 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 -- start start 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.130+0000 7fab3c316b80 1 Processor -- start 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce54b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab3c316b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce59150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab3c316b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5653cce51500 con 0x5653cce367d0 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab3c316b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5653cce50e30 con 0x5653cce370f0 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab3c316b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5653cce51bd0 con 0x5653cce3b580 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab3c316b80 10 client.0 _ll_register_callbacks cb 0x5653ccd21900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce54b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce54b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39676/0 (socket says 172.21.15.47:39676) 2024-04-25T20:59:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 -- 172.21.15.47:0/282160136 learned_addr learned my addr 172.21.15.47:0/282160136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2affd700 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2bfff700 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce59150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 -- 172.21.15.47:0/282160136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 msgr2=0x5653cce59150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce59150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 -- 172.21.15.47:0/282160136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 msgr2=0x5653cce550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 -- 172.21.15.47:0/282160136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cce59690 con 0x5653cce367d0 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce54b70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00a5d0 tx=0x7fab1c030e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942dc0d7beeaa9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.131+0000 7fab0affd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab0affd700 1 -- 172.21.15.47:0/282160136 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c014070 con 0x5653cce367d0 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab0affd700 1 -- 172.21.15.47:0/282160136 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fab1c004cb0 con 0x5653cce367d0 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 1 -- 172.21.15.47:0/282160136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab240a4700 con 0x5653cce367d0 2024-04-25T20:59:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab0affd700 1 -- 172.21.15.47:0/282160136 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab1c021030 con 0x5653cce367d0 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 10 client.15090 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 1 -- 172.21.15.47:0/282160136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653cceb7ab0 con 0x5653cce367d0 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab0affd700 1 -- 172.21.15.47:0/282160136 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab240a4700 con 0x5653cce367d0 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab0affd700 1 -- 172.21.15.47:0/282160136 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fab1c03a070 con 0x5653cce367d0 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 20 client.15090 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5653ccebc660 0x5653ccebb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 4 client.15090 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.132+0000 7fab3c316b80 1 -- 172.21.15.47:0/282160136 --> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] -- command(tid 0: {"prefix": "status"}) -- 0x5653ccd90570 con 0x5653ccebc660 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab2affd700 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5653ccebc660 0x5653ccebb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab2affd700 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5653ccebc660 0x5653ccebb370 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38a9783519809bbd server_cookie=50c944dad86b51c5 in_seq=0 out_seq=0 2024-04-25T20:59:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab0affd700 10 client.15090 ms_handle_connect on v2:172.21.15.92:6836/1984144918 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab0affd700 1 -- 172.21.15.47:0/282160136 <== mds.1 v2:172.21.15.92:6836/1984144918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5653ccd90570 con 0x5653ccebc660 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab0affd700 10 client.15090 handle_command_reply: tid=0 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab3c316b80 1 client.15090 shutdown 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab3c316b80 1 -- 172.21.15.47:0/282160136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 msgr2=0x5653cce54b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab3c316b80 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce54b70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00a5d0 tx=0x7fab1c030e20 comp rx=0 tx=0).stop 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab3c316b80 1 -- 172.21.15.47:0/282160136 shutdown_connections 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.133+0000 7fab3c316b80 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5653ccebc660 0x5653ccebb370 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab3c316b80 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653cce367d0 0x5653cce54b70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab3c316b80 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653cce3b580 0x5653cce59150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab0affd700 0 client.15090 ms_handle_reset on v2:172.21.15.92:6836/1984144918 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab3c316b80 1 --2- 172.21.15.47:0/282160136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653cce370f0 0x5653cce550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab3c316b80 1 -- 172.21.15.47:0/282160136 >> 172.21.15.47:0/282160136 conn(0x5653ccd63640 msgr2=0x5653cce312b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab3c316b80 1 -- 172.21.15.47:0/282160136 shutdown_connections 2024-04-25T20:59:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab3c316b80 1 -- 172.21.15.47:0/282160136 wait complete. 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab3c316b80 20 client.15090 trim_cache size 0 max 16384 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24467, 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 161.43090192700001, 2024-04-25T20:59:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 25, 2024-04-25T20:59:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 91, 2024-04-25T20:59:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T20:59:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 202.20650836600001 2024-04-25T20:59:47.142 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T20:59:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab31a8d700 1 -- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fab140a3cc0 msgr2=0x7fab140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab31a8d700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fab140a3cc0 0x7fab140a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005950 tx=0x7fab1c005f30 comp rx=0 tx=0).stop 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab31a8d700 1 -- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c108a60 msgr2=0x7fab2c102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.134+0000 7fab31a8d700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c108a60 0x7fab2c102430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b970 tx=0x7fab2400bd30 comp rx=0 tx=0).stop 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.135+0000 7fab31a8d700 1 -- 172.21.15.47:0/2462230201 shutdown_connections 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.135+0000 7fab31a8d700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fab140a3cc0 0x7fab140a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.135+0000 7fab31a8d700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab2c108a60 0x7fab2c102430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.135+0000 7fab31a8d700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab2c06b1f0 0x7fab2c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.135+0000 7fab31a8d700 1 --2- 172.21.15.47:0/2462230201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c06bbb0 0x7fab2c101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.135+0000 7fab31a8d700 1 -- 172.21.15.47:0/2462230201 >> 172.21.15.47:0/2462230201 conn(0x7fab2c0789e0 msgr2=0x7fab2c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.135+0000 7fab31a8d700 1 -- 172.21.15.47:0/2462230201 shutdown_connections 2024-04-25T20:59:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:47.135+0000 7fab31a8d700 1 -- 172.21.15.47:0/2462230201 wait complete. 2024-04-25T20:59:47.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:47 smithi047 ceph-mon[29195]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:47.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T20:59:47.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:47 smithi186 ceph-mon[32529]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:48.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:47 smithi092 ceph-mon[32524]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:59:48.087 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 -- 172.21.15.47:0/2718841400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 msgr2=0x7fcc8806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 --2- 172.21.15.47:0/2718841400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 0x7fcc8806abb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009990 tx=0x7fcc7c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 -- 172.21.15.47:0/2718841400 shutdown_connections 2024-04-25T20:59:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 --2- 172.21.15.47:0/2718841400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc8806b0f0 0x7fcc8810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 --2- 172.21.15.47:0/2718841400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc88108a30 0x7fcc88108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 --2- 172.21.15.47:0/2718841400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 0x7fcc8806abb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 -- 172.21.15.47:0/2718841400 >> 172.21.15.47:0/2718841400 conn(0x7fcc880788f0 msgr2=0x7fcc88078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 -- 172.21.15.47:0/2718841400 shutdown_connections 2024-04-25T20:59:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 -- 172.21.15.47:0/2718841400 wait complete. 2024-04-25T20:59:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.745+0000 7fcc8d628700 1 Processor -- start 2024-04-25T20:59:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc8d628700 1 -- start start 2024-04-25T20:59:49.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc8d628700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 0x7fcc88199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc8d628700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc8806b0f0 0x7fcc881996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc8d628700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88108a30 0x7fcc8819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc8d628700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88112c30 con 0x7fcc8806a750 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc8d628700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88112ab0 con 0x7fcc8806b0f0 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc8d628700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88112db0 con 0x7fcc88108a30 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc86ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 0x7fcc88199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc86ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 0x7fcc88199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39698/0 (socket says 172.21.15.47:39698) 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc86ffd700 1 -- 172.21.15.47:0/3720594166 learned_addr learned my addr 172.21.15.47:0/3720594166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc867fc700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc8806b0f0 0x7fcc881996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc86ffd700 1 -- 172.21.15.47:0/3720594166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88108a30 msgr2=0x7fcc8819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88108a30 0x7fcc8819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc86ffd700 1 -- 172.21.15.47:0/3720594166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc8806b0f0 msgr2=0x7fcc881996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc8806b0f0 0x7fcc881996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc86ffd700 1 -- 172.21.15.47:0/3720594166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8819e190 con 0x7fcc8806a750 2024-04-25T20:59:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.746+0000 7fcc867fc700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc8806b0f0 0x7fcc881996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.747+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 0x7fcc88199180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800ba50 tx=0x7fcc7800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.747+0000 7fcc6ffff700 1 -- 172.21.15.47:0/3720594166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78019070 con 0x7fcc8806a750 2024-04-25T20:59:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.747+0000 7fcc8d628700 1 -- 172.21.15.47:0/3720594166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8819e480 con 0x7fcc8806a750 2024-04-25T20:59:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.747+0000 7fcc6ffff700 1 -- 172.21.15.47:0/3720594166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc78014440 con 0x7fcc8806a750 2024-04-25T20:59:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.747+0000 7fcc6ffff700 1 -- 172.21.15.47:0/3720594166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc780075d0 con 0x7fcc8806a750 2024-04-25T20:59:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.747+0000 7fcc8d628700 1 -- 172.21.15.47:0/3720594166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc881a5d00 con 0x7fcc8806a750 2024-04-25T20:59:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.750+0000 7fcc6ffff700 1 -- 172.21.15.47:0/3720594166 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcc7801c070 con 0x7fcc8806a750 2024-04-25T20:59:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.750+0000 7fcc6ffff700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fcc700a3c40 0x7fcc700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc867fc700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fcc700a3c40 0x7fcc700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc6ffff700 1 -- 172.21.15.47:0/3720594166 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fcc780cf8e0 con 0x7fcc8806a750 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc97eb1b80 1 Processor -- start 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc97eb1b80 1 -- start start 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc867fc700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fcc700a3c40 0x7fcc700a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c006010 tx=0x7fcc7c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f47d0 0x55a6f50f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f50f0 0x55a6f50f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5046ce0 con 0x55a6f50f47d0 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a6f4fed910 con 0x55a6f50f9580 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5042ca0 con 0x55a6f50f50f0 2024-04-25T20:59:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc86ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f47d0 0x55a6f50f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc86ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f47d0 0x55a6f50f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39708/0 (socket says 172.21.15.47:39708) 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.751+0000 7fcc86ffd700 1 -- 172.21.15.47:0/3208560745 learned_addr learned my addr 172.21.15.47:0/3208560745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc877fe700 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc86ffd700 1 -- 172.21.15.47:0/3208560745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f50f0 msgr2=0x55a6f50f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f50f0 0x55a6f50f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc86ffd700 1 -- 172.21.15.47:0/3208560745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 msgr2=0x55a6f50f99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50f99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc86ffd700 1 -- 172.21.15.47:0/3208560745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f4f94880 con 0x55a6f50f47d0 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc877fe700 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f47d0 0x55a6f50f4bb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcc780a4290 tx=0x7fcc78004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665a4fa1ccdc5a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc6d7fa700 1 -- 172.21.15.47:0/3208560745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7809ff00 con 0x55a6f50f47d0 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc6d7fa700 1 -- 172.21.15.47:0/3208560745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc780a47c0 con 0x55a6f50f47d0 2024-04-25T20:59:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.752+0000 7fcc6d7fa700 1 -- 172.21.15.47:0/3208560745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7809c010 con 0x55a6f50f47d0 2024-04-25T20:59:49.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3208560745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f47d0 msgr2=0x55a6f50f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f47d0 0x55a6f50f4bb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcc780a4290 tx=0x7fcc78004be0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3208560745 shutdown_connections 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f47d0 0x55a6f50f4bb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/3208560745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f50f0 0x55a6f50f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3208560745 >> 172.21.15.47:0/3208560745 conn(0x55a6f5021640 msgr2=0x55a6f50ef820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3208560745 shutdown_connections 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3208560745 wait complete. 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 -- start start 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 Processor -- start 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 0x55a6f5100b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50fde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50fe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.753+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5101610 con 0x55a6f50f50f0 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a6f4fe5b80 con 0x55a6f50f9580 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5101790 con 0x55a6f50f47d0 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc97eb1b80 10 client.0 _ll_register_callbacks cb 0x55a6f4fdf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50fde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39716/0 (socket says 172.21.15.47:39716) 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 -- 172.21.15.47:0/373621392 learned_addr learned my addr 172.21.15.47:0/373621392 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 0x55a6f5100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc877fe700 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 -- 172.21.15.47:0/373621392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 msgr2=0x55a6f5100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 0x55a6f5100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 -- 172.21.15.47:0/373621392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 msgr2=0x55a6f50fe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50fe390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 -- 172.21.15.47:0/373621392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f4ffe220 con 0x55a6f50f50f0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc867fc700 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50fde50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009750 tx=0x7fcc7c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f435ee52308ee7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc6aff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc6aff5700 1 -- 172.21.15.47:0/373621392 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c003a70 con 0x55a6f50f50f0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc6aff5700 1 -- 172.21.15.47:0/373621392 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fcc7c003c10 con 0x55a6f50f50f0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.754+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/373621392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc780a47c0 con 0x55a6f50f50f0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc6aff5700 1 -- 172.21.15.47:0/373621392 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc7c003e10 con 0x55a6f50f50f0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc97eb1b80 10 client.15102 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/373621392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6f510bf00 con 0x55a6f50f50f0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc6aff5700 1 -- 172.21.15.47:0/373621392 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc780a47c0 con 0x55a6f50f50f0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc6aff5700 1 -- 172.21.15.47:0/373621392 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcc7c038050 con 0x55a6f50f50f0 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc97eb1b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc97eb1b80 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc97eb1b80 20 client.15102 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55a6f510e4d0 0x55a6f510f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc97eb1b80 4 client.15102 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/373621392 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f50544d0 con 0x55a6f510e4d0 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.755+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55a6f510e4d0 0x55a6f510f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.756+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55a6f510e4d0 0x55a6f510f940 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a11cb734453cd7 server_cookie=d0620fee6a855b9 in_seq=0 out_seq=0 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.756+0000 7fcc6aff5700 10 client.15102 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.756+0000 7fcc6aff5700 1 -- 172.21.15.47:0/373621392 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6f50544d0 con 0x55a6f510e4d0 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.756+0000 7fcc6aff5700 10 client.15102 handle_command_reply: tid=0 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.756+0000 7fcc97eb1b80 1 client.15102 shutdown 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/373621392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 msgr2=0x55a6f50fde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50fde50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009750 tx=0x7fcc7c009310 comp rx=0 tx=0).stop 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/373621392 shutdown_connections 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55a6f510e4d0 0x55a6f510f940 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc6aff5700 0 client.15102 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T20:59:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f50fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 0x55a6f5100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/373621392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50fde50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/373621392 >> 172.21.15.47:0/373621392 conn(0x55a6f5021640 msgr2=0x55a6f50fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/373621392 shutdown_connections 2024-04-25T20:59:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/373621392 wait complete. 2024-04-25T20:59:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.757+0000 7fcc97eb1b80 20 client.15102 trim_cache size 0 max 16384 2024-04-25T20:59:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.777+0000 7fcc97eb1b80 1 Processor -- start 2024-04-25T20:59:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.777+0000 7fcc97eb1b80 1 -- start start 2024-04-25T20:59:49.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:49 smithi047 ceph-mon[29195]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f47d0 0x55a6f50fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f9580 0x55a6f50f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a6f510dc80 con 0x55a6f50f50f0 2024-04-25T20:59:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a6f510e350 con 0x55a6f50f47d0 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a6f510ea20 con 0x55a6f50f9580 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39728/0 (socket says 172.21.15.47:39728) 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 -- 172.21.15.47:0/3704545853 learned_addr learned my addr 172.21.15.47:0/3704545853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f47d0 0x55a6f50fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc877fe700 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f9580 0x55a6f50f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 -- 172.21.15.47:0/3704545853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f9580 msgr2=0x55a6f50f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f9580 0x55a6f50f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 -- 172.21.15.47:0/3704545853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f47d0 msgr2=0x55a6f50fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f47d0 0x55a6f50fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 -- 172.21.15.47:0/3704545853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f5101740 con 0x55a6f50f50f0 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc867fc700 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50eb700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c005740 tx=0x7fcc7c02f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895215d4b2e8656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.778+0000 7fcc6cff9700 1 -- 172.21.15.47:0/3704545853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c003a70 con 0x55a6f50f50f0 2024-04-25T20:59:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc6cff9700 1 -- 172.21.15.47:0/3704545853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc7c003c10 con 0x55a6f50f50f0 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc6cff9700 1 -- 172.21.15.47:0/3704545853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c00f740 con 0x55a6f50f50f0 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3704545853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 msgr2=0x55a6f50eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50eb700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c005740 tx=0x7fcc7c02f4f0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3704545853 shutdown_connections 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f9580 0x55a6f50f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f47d0 0x55a6f50fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/3704545853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f50f50f0 0x55a6f50eb700 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3704545853 >> 172.21.15.47:0/3704545853 conn(0x55a6f5021640 msgr2=0x55a6f50f35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3704545853 shutdown_connections 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.779+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/3704545853 wait complete. 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 1 -- start start 2024-04-25T20:59:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 1 Processor -- start 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 0x55a6f5118dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f5119300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f510e4d0 0x55a6f5113d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a6f510f7c0 con 0x55a6f510e4d0 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a6f510f0f0 con 0x55a6f50f9580 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a6f510fe90 con 0x55a6f50f47d0 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc97eb1b80 10 client.0 _ll_register_callbacks cb 0x55a6f4fdf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc877fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f510e4d0 0x55a6f5113d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc877fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f510e4d0 0x55a6f5113d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39742/0 (socket says 172.21.15.47:39742) 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc877fe700 1 -- 172.21.15.47:0/1712610963 learned_addr learned my addr 172.21.15.47:0/1712610963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 0x55a6f5118dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.780+0000 7fcc867fc700 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f5119300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc877fe700 1 -- 172.21.15.47:0/1712610963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 msgr2=0x55a6f5118dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc877fe700 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 0x55a6f5118dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc877fe700 1 -- 172.21.15.47:0/1712610963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 msgr2=0x55a6f5119300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc877fe700 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f5119300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc877fe700 1 -- 172.21.15.47:0/1712610963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f51144f0 con 0x55a6f510e4d0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc877fe700 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f510e4d0 0x55a6f5113d70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009c00 tx=0x7fcc80009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15a217b6333a9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc6aff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc6aff5700 1 -- 172.21.15.47:0/1712610963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80012dd0 con 0x55a6f510e4d0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc6aff5700 1 -- 172.21.15.47:0/1712610963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fcc8001a3e0 con 0x55a6f510e4d0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/1712610963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc7c003c10 con 0x55a6f510e4d0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc6aff5700 1 -- 172.21.15.47:0/1712610963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc8001a5e0 con 0x55a6f510e4d0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc97eb1b80 10 client.15108 fetch_fsmap learned FSMap version 25 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.781+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/1712610963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6f51760e0 con 0x55a6f510e4d0 2024-04-25T20:59:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc6aff5700 1 -- 172.21.15.47:0/1712610963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc7c003c10 con 0x55a6f510e4d0 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc6aff5700 1 -- 172.21.15.47:0/1712610963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcc80024050 con 0x55a6f510e4d0 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc97eb1b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc97eb1b80 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc97eb1b80 20 client.15108 populate_metadata read hostname 'smithi047' 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55a6f517ad20 0x55a6f5179a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc97eb1b80 4 client.15108 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/1712610963 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "status"}) -- 0x55a6f504e570 con 0x55a6f517ad20 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.782+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55a6f517ad20 0x55a6f5179a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc86ffd700 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55a6f517ad20 0x55a6f5179a30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4e512d782e1d50 server_cookie=f48b36c3ed47eda2 in_seq=0 out_seq=0 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc6aff5700 10 client.15108 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc6aff5700 1 -- 172.21.15.47:0/1712610963 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a6f504e570 con 0x55a6f517ad20 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc6aff5700 10 client.15108 handle_command_reply: tid=0 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 client.15108 shutdown 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/1712610963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f510e4d0 msgr2=0x55a6f5113d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f510e4d0 0x55a6f5113d70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009c00 tx=0x7fcc80009f10 comp rx=0 tx=0).stop 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/1712610963 shutdown_connections 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55a6f517ad20 0x55a6f5179a30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6f510e4d0 0x55a6f5113d70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6f50f47d0 0x55a6f5118dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 --2- 172.21.15.47:0/1712610963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6f50f9580 0x55a6f5119300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/1712610963 >> 172.21.15.47:0/1712610963 conn(0x55a6f5021640 msgr2=0x55a6f50f3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc6aff5700 0 client.15108 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/1712610963 shutdown_connections 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 1 -- 172.21.15.47:0/1712610963 wait complete. 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.783+0000 7fcc97eb1b80 20 client.15108 trim_cache size 0 max 16384 2024-04-25T20:59:49.789 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24488, 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 176.50657869400001, 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 25, 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 91, 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 192.486590304 2024-04-25T20:59:49.790 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 -- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fcc700a3c40 msgr2=0x7fcc700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fcc700a3c40 0x7fcc700a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c006010 tx=0x7fcc7c00b540 comp rx=0 tx=0).stop 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 -- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 msgr2=0x7fcc88199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 0x7fcc88199180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800ba50 tx=0x7fcc7800be10 comp rx=0 tx=0).stop 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 -- 172.21.15.47:0/3720594166 shutdown_connections 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7fcc700a3c40 0x7fcc700a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc8806a750 0x7fcc88199180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88108a30 0x7fcc8819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 --2- 172.21.15.47:0/3720594166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc8806b0f0 0x7fcc881996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 -- 172.21.15.47:0/3720594166 >> 172.21.15.47:0/3720594166 conn(0x7fcc880788f0 msgr2=0x7fcc880ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 -- 172.21.15.47:0/3720594166 shutdown_connections 2024-04-25T20:59:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:49.784+0000 7fcc8d628700 1 -- 172.21.15.47:0/3720594166 wait complete. 2024-04-25T20:59:49.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:49 smithi186 ceph-mon[32529]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:50.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:49 smithi092 ceph-mon[32524]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:50.512 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T20:59:50.529 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T20:59:50.529 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T20:59:50.529 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T20:59:50.529 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T20:59:50.530 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T20:59:50.530 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T20:59:50.531 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T20:59:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 -- 172.21.15.47:0/2137695743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc100e60 msgr2=0x7f34bc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 --2- 172.21.15.47:0/2137695743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc100e60 0x7f34bc101280 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009900 tx=0x7f34a402a410 comp rx=0 tx=0).stop 2024-04-25T20:59:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 -- 172.21.15.47:0/2137695743 shutdown_connections 2024-04-25T20:59:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 --2- 172.21.15.47:0/2137695743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc102a00 0x7f34bc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 --2- 172.21.15.47:0/2137695743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34bc102040 0x7f34bc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 --2- 172.21.15.47:0/2137695743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc100e60 0x7f34bc101280 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 -- 172.21.15.47:0/2137695743 >> 172.21.15.47:0/2137695743 conn(0x7f34bc0fc440 msgr2=0x7f34bc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 -- 172.21.15.47:0/2137695743 shutdown_connections 2024-04-25T20:59:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.644+0000 7f34c0d95640 1 -- 172.21.15.47:0/2137695743 wait complete. 2024-04-25T20:59:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34c0d95640 1 Processor -- start 2024-04-25T20:59:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34c0d95640 1 -- start start 2024-04-25T20:59:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34c0d95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34bc100e60 0x7f34bc06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34c0d95640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc102040 0x7f34bc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34c0d95640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc102a00 0x7f34bc19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34c0d95640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc10c9f0 con 0x7f34bc102040 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34c0d95640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc10c870 con 0x7f34bc100e60 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34c0d95640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc10cb70 con 0x7f34bc102a00 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc102040 0x7f34bc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc102040 0x7f34bc19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39760/0 (socket says 172.21.15.47:39760) 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 -- 172.21.15.47:0/4224364819 learned_addr learned my addr 172.21.15.47:0/4224364819 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34bad9d640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc102a00 0x7f34bc19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34ba59c640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34bc100e60 0x7f34bc06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 -- 172.21.15.47:0/4224364819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc102a00 msgr2=0x7f34bc19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc102a00 0x7f34bc19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 -- 172.21.15.47:0/4224364819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34bc100e60 msgr2=0x7f34bc06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34bc100e60 0x7f34bc06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 -- 172.21.15.47:0/4224364819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc19fd90 con 0x7f34bc102040 2024-04-25T20:59:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34bad9d640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc102a00 0x7f34bc19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.645+0000 7f34b9d9b640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc102040 0x7f34bc19ce60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00e9f0 tx=0x7f34ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.646+0000 7f34b37fe640 1 -- 172.21.15.47:0/4224364819 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00cd80 con 0x7f34bc102040 2024-04-25T20:59:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.646+0000 7f34b37fe640 1 -- 172.21.15.47:0/4224364819 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ac0108e0 con 0x7f34bc102040 2024-04-25T20:59:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.646+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc19ffc0 con 0x7f34bc102040 2024-04-25T20:59:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.646+0000 7f34b37fe640 1 -- 172.21.15.47:0/4224364819 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac018980 con 0x7f34bc102040 2024-04-25T20:59:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.646+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1a04e0 con 0x7f34bc102040 2024-04-25T20:59:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.647+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc108200 con 0x7f34bc102040 2024-04-25T20:59:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.648+0000 7f34b37fe640 1 -- 172.21.15.47:0/4224364819 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f34ac010430 con 0x7f34bc102040 2024-04-25T20:59:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.649+0000 7f34b37fe640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f34a80a3c50 0x7f34a80a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.649+0000 7f34ba59c640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f34a80a3c50 0x7f34a80a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.649+0000 7f34b37fe640 1 -- 172.21.15.47:0/4224364819 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f34ac0d0820 con 0x7f34bc102040 2024-04-25T20:59:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.649+0000 7f34ba59c640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f34a80a3c50 0x7f34a80a6110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34a4000c00 tx=0x7f34a40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.653+0000 7f34b37fe640 1 -- 172.21.15.47:0/4224364819 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34ac09d1f0 con 0x7f34bc102040 2024-04-25T20:59:50.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:50.895+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f34bc062bf0 con 0x7f34bc102040 2024-04-25T20:59:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.663+0000 7f34b37fe640 1 -- 172.21.15.47:0/4224364819 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f34ac09caf0 con 0x7f34bc102040 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f34a80a3c50 msgr2=0x7f34a80a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f34a80a3c50 0x7f34a80a6110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34a4000c00 tx=0x7f34a40314a0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc102040 msgr2=0x7f34bc19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc102040 0x7f34bc19ce60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00e9f0 tx=0x7f34ac00eeb0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 shutdown_connections 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f34a80a3c50 0x7f34a80a6110 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc102a00 0x7f34bc19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34bc102040 0x7f34bc19ce60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 --2- 172.21.15.47:0/4224364819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34bc100e60 0x7f34bc06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 >> 172.21.15.47:0/4224364819 conn(0x7f34bc0fc440 msgr2=0x7f34bc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:51.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 shutdown_connections 2024-04-25T20:59:51.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.666+0000 7f34c0d95640 1 -- 172.21.15.47:0/4224364819 wait complete. 2024-04-25T20:59:51.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T20:59:51.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 -- 172.21.15.47:0/1738519125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 msgr2=0x7f84ac10a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:51.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 --2- 172.21.15.47:0/1738519125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 0x7f84ac10a100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009ca0 tx=0x7f84a402a8c0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 -- 172.21.15.47:0/1738519125 shutdown_connections 2024-04-25T20:59:51.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 --2- 172.21.15.47:0/1738519125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 0x7f84ac10a100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 --2- 172.21.15.47:0/1738519125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac1053e0 0x7f84ac1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 --2- 172.21.15.47:0/1738519125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84ac102ab0 0x7f84ac104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 -- 172.21.15.47:0/1738519125 >> 172.21.15.47:0/1738519125 conn(0x7f84ac0fc420 msgr2=0x7f84ac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:51.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 -- 172.21.15.47:0/1738519125 shutdown_connections 2024-04-25T20:59:51.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.797+0000 7f84b182a640 1 -- 172.21.15.47:0/1738519125 wait complete. 2024-04-25T20:59:51.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84b182a640 1 Processor -- start 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84b182a640 1 -- start start 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84b182a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84ac102ab0 0x7f84ac198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84b182a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac1053e0 0x7f84ac198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84b182a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 0x7f84ac19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84b182a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac10daf0 con 0x7f84ac107d10 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84b182a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac10d970 con 0x7f84ac1053e0 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84b182a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac10dc70 con 0x7f84ac102ab0 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 0x7f84ac19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84aaffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84ac102ab0 0x7f84ac198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:51.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 0x7f84ac19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39788/0 (socket says 172.21.15.47:39788) 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84ab7fe640 1 -- 172.21.15.47:0/1642788610 learned_addr learned my addr 172.21.15.47:0/1642788610 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.798+0000 7f84aa7fc640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac1053e0 0x7f84ac198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84aaffd640 1 -- 172.21.15.47:0/1642788610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac1053e0 msgr2=0x7f84ac198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84aaffd640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac1053e0 0x7f84ac198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84aaffd640 1 -- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 msgr2=0x7f84ac19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84aaffd640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 0x7f84ac19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84aaffd640 1 -- 172.21.15.47:0/1642788610 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac19fd60 con 0x7f84ac102ab0 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84ab7fe640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 0x7f84ac19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84aa7fc640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac1053e0 0x7f84ac198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84aaffd640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84ac102ab0 0x7f84ac198140 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f849c00e9f0 tx=0x7f849c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84b0828640 1 -- 172.21.15.47:0/1642788610 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c00ccc0 con 0x7f84ac102ab0 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84b0828640 1 -- 172.21.15.47:0/1642788610 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f849c005020 con 0x7f84ac102ab0 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ac19fff0 con 0x7f84ac102ab0 2024-04-25T20:59:51.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84b0828640 1 -- 172.21.15.47:0/1642788610 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c004a00 con 0x7f84ac102ab0 2024-04-25T20:59:51.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.799+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ac1a0460 con 0x7f84ac102ab0 2024-04-25T20:59:51.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.800+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ac0fdf10 con 0x7f84ac102ab0 2024-04-25T20:59:51.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.801+0000 7f84b0828640 1 -- 172.21.15.47:0/1642788610 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f849c004ba0 con 0x7f84ac102ab0 2024-04-25T20:59:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.802+0000 7f84b0828640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f84940a3ca0 0x7f84940a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.802+0000 7f84aa7fc640 1 -- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f84940a3ca0 msgr2=0x7f84940a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.802+0000 7f84b0828640 1 -- 172.21.15.47:0/1642788610 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f849c0d03f0 con 0x7f84ac102ab0 2024-04-25T20:59:51.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.802+0000 7f84aa7fc640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f84940a3ca0 0x7f84940a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:59:51.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.807+0000 7f84b0828640 1 -- 172.21.15.47:0/1642788610 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f849c09cdc0 con 0x7f84ac102ab0 2024-04-25T20:59:51.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:51 smithi186 ceph-mon[32529]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:51.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:51 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/4224364819' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T20:59:51.971 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: ignoring --setuser ceph since I am not root 2024-04-25T20:59:51.971 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: ignoring --setgroup ceph since I am not root 2024-04-25T20:59:51.971 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:51 smithi186 ceph-mgr[34325]: -- 172.21.15.186:0/3712102435 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55908fcea5a0 con 0x55908fcbec00 2024-04-25T20:59:51.971 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:51.847+0000 7f2434259200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:59:51.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:51.996+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f84ac02d5c0 con 0x7f84ac102ab0 2024-04-25T20:59:52.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.003+0000 7f84aa7fc640 1 -- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f84940a3ca0 msgr2=0x7f84940a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:52.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.003+0000 7f84aa7fc640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f84940a3ca0 0x7f84940a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:59:52.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.003+0000 7f84b0828640 1 -- 172.21.15.47:0/1642788610 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) ==== 138+0+0 (secure 0 0 0) 0x7f849c09c6c0 con 0x7f84ac102ab0 2024-04-25T20:59:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f84940a3ca0 msgr2=0x7f84940a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:59:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f84940a3ca0 0x7f84940a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84ac102ab0 msgr2=0x7f84ac198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84ac102ab0 0x7f84ac198140 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f849c00e9f0 tx=0x7f849c00eeb0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 shutdown_connections 2024-04-25T20:59:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f84940a3ca0 0x7f84940a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84ac107d10 0x7f84ac19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac1053e0 0x7f84ac198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 --2- 172.21.15.47:0/1642788610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84ac102ab0 0x7f84ac198140 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 >> 172.21.15.47:0/1642788610 conn(0x7f84ac0fc420 msgr2=0x7f84ac1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:52.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 shutdown_connections 2024-04-25T20:59:52.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.005+0000 7f84b182a640 1 -- 172.21.15.47:0/1642788610 wait complete. 2024-04-25T20:59:52.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:51 smithi047 ceph-mon[29195]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:52.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:51 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/4224364819' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T20:59:52.017 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: ignoring --setuser ceph since I am not root 2024-04-25T20:59:52.017 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: ignoring --setgroup ceph since I am not root 2024-04-25T20:59:52.017 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:51.861+0000 7f143c2cd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:59:52.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T20:59:52.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:51 smithi092 ceph-mon[32524]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:59:52.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:51 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/4224364819' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T20:59:52.029 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:51 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: ignoring --setuser ceph since I am not root 2024-04-25T20:59:52.029 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:51 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: ignoring --setgroup ceph since I am not root 2024-04-25T20:59:52.029 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:51 smithi092 ceph-mgr[33949]: -- 172.21.15.92:0/1656529011 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558decdda5a0 con 0x558decdae800 2024-04-25T20:59:52.029 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:51 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:51.845+0000 7f0e6418a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:59:52.029 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:52.028+0000 7f0e6418a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:59:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.121+0000 7f4b61c33640 1 -- 172.21.15.47:0/4195520694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c100e60 msgr2=0x7f4b5c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.121+0000 7f4b61c33640 1 --2- 172.21.15.47:0/4195520694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c100e60 0x7f4b5c101280 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009900 tx=0x7f4b5002a410 comp rx=0 tx=0).stop 2024-04-25T20:59:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.121+0000 7f4b61c33640 1 -- 172.21.15.47:0/4195520694 shutdown_connections 2024-04-25T20:59:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.121+0000 7f4b61c33640 1 --2- 172.21.15.47:0/4195520694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b5c102a00 0x7f4b5c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.121+0000 7f4b61c33640 1 --2- 172.21.15.47:0/4195520694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b5c102040 0x7f4b5c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.121+0000 7f4b61c33640 1 --2- 172.21.15.47:0/4195520694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c100e60 0x7f4b5c101280 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.121+0000 7f4b61c33640 1 -- 172.21.15.47:0/4195520694 >> 172.21.15.47:0/4195520694 conn(0x7f4b5c0fc420 msgr2=0x7f4b5c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.122+0000 7f4b61c33640 1 -- 172.21.15.47:0/4195520694 shutdown_connections 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.122+0000 7f4b61c33640 1 -- 172.21.15.47:0/4195520694 wait complete. 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.122+0000 7f4b61c33640 1 Processor -- start 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.122+0000 7f4b61c33640 1 -- start start 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.122+0000 7f4b61c33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b5c100e60 0x7f4b5c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.122+0000 7f4b61c33640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b5c102040 0x7f4b5c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.122+0000 7f4b61c33640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c102a00 0x7f4b5c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.122+0000 7f4b61c33640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10ca20 con 0x7f4b5c102a00 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b61c33640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10c8a0 con 0x7f4b5c100e60 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b61c33640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10cba0 con 0x7f4b5c102040 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c102a00 0x7f4b5c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b5c100e60 0x7f4b5c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c102a00 0x7f4b5c19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39796/0 (socket says 172.21.15.47:39796) 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5affd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b5c102040 0x7f4b5c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 -- 172.21.15.47:0/3647577305 learned_addr learned my addr 172.21.15.47:0/3647577305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 -- 172.21.15.47:0/3647577305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b5c102040 msgr2=0x7f4b5c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b5c102040 0x7f4b5c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 -- 172.21.15.47:0/3647577305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b5c100e60 msgr2=0x7f4b5c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b5c100e60 0x7f4b5c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 -- 172.21.15.47:0/3647577305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c19fdd0 con 0x7f4b5c102a00 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5b7fe640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b5c100e60 0x7f4b5c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b5bfff640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c102a00 0x7f4b5c19f760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009fd0 tx=0x7f4b5400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b58ff9640 1 -- 172.21.15.47:0/3647577305 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54017070 con 0x7f4b5c102a00 2024-04-25T20:59:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c1a00c0 con 0x7f4b5c102a00 2024-04-25T20:59:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b58ff9640 1 -- 172.21.15.47:0/3647577305 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b54003e20 con 0x7f4b5c102a00 2024-04-25T20:59:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b58ff9640 1 -- 172.21.15.47:0/3647577305 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54010460 con 0x7f4b5c102a00 2024-04-25T20:59:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.123+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1a05c0 con 0x7f4b5c102a00 2024-04-25T20:59:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.124+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c1a0820 con 0x7f4b5c102a00 2024-04-25T20:59:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.126+0000 7f4b58ff9640 1 -- 172.21.15.47:0/3647577305 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4b54003970 con 0x7f4b5c102a00 2024-04-25T20:59:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.127+0000 7f4b58ff9640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4b440a3ca0 0x7f4b440a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.127+0000 7f4b5b7fe640 1 -- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4b440a3ca0 msgr2=0x7f4b440a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.127+0000 7f4b58ff9640 1 -- 172.21.15.47:0/3647577305 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f4b540d00c0 con 0x7f4b5c102a00 2024-04-25T20:59:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.127+0000 7f4b5b7fe640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4b440a3ca0 0x7f4b440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:59:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.131+0000 7f4b58ff9640 1 -- 172.21.15.47:0/3647577305 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b5409c3e0 con 0x7f4b5c102a00 2024-04-25T20:59:52.282 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:52.131+0000 7f0e6418a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:59:52.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.318+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4b5c1a09f0 con 0x7f4b5c102a00 2024-04-25T20:59:52.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.322+0000 7f4b58ff9640 1 -- 172.21.15.47:0/3647577305 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) ==== 133+0+0 (secure 0 0 0) 0x7f4b540a06e0 con 0x7f4b5c102a00 2024-04-25T20:59:52.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.324+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4b440a3ca0 msgr2=0x7f4b440a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:59:52.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.324+0000 7f4b61c33640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4b440a3ca0 0x7f4b440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.324+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c102a00 msgr2=0x7f4b5c19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:52.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.324+0000 7f4b61c33640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c102a00 0x7f4b5c19f760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009fd0 tx=0x7f4b5400b260 comp rx=0 tx=0).stop 2024-04-25T20:59:52.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.325+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 shutdown_connections 2024-04-25T20:59:52.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.325+0000 7f4b61c33640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f4b440a3ca0 0x7f4b440a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.325+0000 7f4b61c33640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b5c102a00 0x7f4b5c19f760 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.325+0000 7f4b61c33640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b5c102040 0x7f4b5c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.325+0000 7f4b61c33640 1 --2- 172.21.15.47:0/3647577305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b5c100e60 0x7f4b5c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:52.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.325+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 >> 172.21.15.47:0/3647577305 conn(0x7f4b5c0fc420 msgr2=0x7f4b5c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:52.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.325+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 shutdown_connections 2024-04-25T20:59:52.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:52.325+0000 7f4b61c33640 1 -- 172.21.15.47:0/3647577305 wait complete. 2024-04-25T20:59:52.336 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:52.056+0000 7f143c2cd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:59:52.336 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:52.168+0000 7f143c2cd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:59:52.336 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T20:59:52.365 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T20:59:52.365 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T20:59:52.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:52.365 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-25T20:59:52.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:52.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-25T20:59:52.451 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:52.033+0000 7f2434259200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:59:52.451 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:52.136+0000 7f2434259200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:59:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:Cannot find device "ceph-brx" 2024-04-25T20:59:52.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:59:52.452 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T20:59:52.453 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T20:59:52.453 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T20:59:52.453 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T20:59:52.590 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:52.497+0000 7f143c2cd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:59:52.590 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:52.589+0000 7f143c2cd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:59:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:52 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/4224364819' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T20:59:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:52 smithi186 ceph-mon[32529]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-25T20:59:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:52 smithi186 ceph-mon[32529]: from='client.? ' entity='client.admin' 2024-04-25T20:59:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:52 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3647577305' entity='client.admin' 2024-04-25T20:59:52.721 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:52.449+0000 7f2434259200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:59:52.721 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:52.534+0000 7f2434259200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:59:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:52.782 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:52.438+0000 7f0e6418a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:59:52.782 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:52.523+0000 7f0e6418a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:59:52.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:52 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/4224364819' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T20:59:52.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:52 smithi092 ceph-mon[32524]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-25T20:59:52.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:52 smithi092 ceph-mon[32524]: from='client.? ' entity='client.admin' 2024-04-25T20:59:52.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:52 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3647577305' entity='client.admin' 2024-04-25T20:59:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:52 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/4224364819' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T20:59:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:52 smithi047 ceph-mon[29195]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-25T20:59:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:52 smithi047 ceph-mon[29195]: from='client.? ' entity='client.admin' 2024-04-25T20:59:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:52 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3647577305' entity='client.admin' 2024-04-25T20:59:54.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:53 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:53.701+0000 7f0e6418a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:59:54.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:53 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:53.871+0000 7f0e6418a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:59:54.070 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:53.726+0000 7f2434259200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:59:54.070 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:53.897+0000 7f2434259200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:59:54.170 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:53 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:53.863+0000 7f143c2cd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:59:54.170 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:54.047+0000 7f143c2cd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:59:54.331 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:54.041+0000 7f0e6418a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:59:54.363 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:54.070+0000 7f2434259200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:59:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1736750778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 msgr2=0x7f46a4108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1736750778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 0x7f46a4108e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4694009990 tx=0x7f4694009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1736750778 shutdown_connections 2024-04-25T20:59:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1736750778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 0x7f46a4108e90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1736750778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a4103480 0x7f46a410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1736750778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46a4102ac0 0x7f46a4102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1736750778 >> 172.21.15.47:0/1736750778 conn(0x7f46a4079bf0 msgr2=0x7f46a407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1736750778 shutdown_connections 2024-04-25T20:59:54.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.393+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1736750778 wait complete. 2024-04-25T20:59:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46aa4b2700 1 Processor -- start 2024-04-25T20:59:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46aa4b2700 1 -- start start 2024-04-25T20:59:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46aa4b2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a4102ac0 0x7f46a4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46aa4b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46a4103480 0x7f46a41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46aa4b2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 0x7f46a419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46aa4b2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4112cb0 con 0x7f46a4108ab0 2024-04-25T20:59:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46aa4b2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4112b30 con 0x7f46a4103480 2024-04-25T20:59:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46aa4b2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4112e30 con 0x7f46a4102ac0 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46a8a4f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 0x7f46a419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a4102ac0 0x7f46a4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46a8a4f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 0x7f46a419da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35074/0 (socket says 172.21.15.47:35074) 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46a8a4f700 1 -- 172.21.15.47:0/1443010188 learned_addr learned my addr 172.21.15.47:0/1443010188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.394+0000 7f46a37fe700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46a4103480 0x7f46a41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a8a4f700 1 -- 172.21.15.47:0/1443010188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a4102ac0 msgr2=0x7f46a4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a8a4f700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a4102ac0 0x7f46a4199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a8a4f700 1 -- 172.21.15.47:0/1443010188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46a4103480 msgr2=0x7f46a41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a8a4f700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46a4103480 0x7f46a41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a8a4f700 1 -- 172.21.15.47:0/1443010188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a419e170 con 0x7f46a4108ab0 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a3fff700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a4102ac0 0x7f46a4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a37fe700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46a4103480 0x7f46a41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a8a4f700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 0x7f46a419da30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f469c00ef30 tx=0x7f469c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a17fa700 1 -- 172.21.15.47:0/1443010188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0092e0 con 0x7f46a4108ab0 2024-04-25T20:59:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a419e400 con 0x7f46a4108ab0 2024-04-25T20:59:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a17fa700 1 -- 172.21.15.47:0/1443010188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c004bb0 con 0x7f46a4108ab0 2024-04-25T20:59:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46a17fa700 1 -- 172.21.15.47:0/1443010188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c004020 con 0x7f46a4108ab0 2024-04-25T20:59:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.395+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a41a5ce0 con 0x7f46a4108ab0 2024-04-25T20:59:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.396+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a41041f0 con 0x7f46a4108ab0 2024-04-25T20:59:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.397+0000 7f46a17fa700 1 -- 172.21.15.47:0/1443010188 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f469c01b030 con 0x7f46a4108ab0 2024-04-25T20:59:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.398+0000 7f46a17fa700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f46900a3c90 0x7f46900a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.398+0000 7f46a3fff700 1 -- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f46900a3c90 msgr2=0x7f46900a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.398+0000 7f46a17fa700 1 -- 172.21.15.47:0/1443010188 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f469c0cfe50 con 0x7f46a4108ab0 2024-04-25T20:59:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.398+0000 7f46a3fff700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f46900a3c90 0x7f46900a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:59:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.403+0000 7f46a17fa700 1 -- 172.21.15.47:0/1443010188 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f469c022020 con 0x7f46a4108ab0 2024-04-25T20:59:54.544 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:54.232+0000 7f143c2cd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:59:54.544 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:54.543+0000 7f143c2cd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:59:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.599+0000 7f46a3fff700 1 -- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f46900a3c90 msgr2=0x7f46900a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.599+0000 7f46a3fff700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f46900a3c90 0x7f46900a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:59:54.625 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:54.330+0000 7f0e6418a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:59:54.625 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:54.423+0000 7f0e6418a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:59:54.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.661+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f46a402df80 con 0x7f46a4108ab0 2024-04-25T20:59:54.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.661+0000 7f46a17fa700 1 -- 172.21.15.47:0/1443010188 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f469c010040 con 0x7f46a4108ab0 2024-04-25T20:59:54.662 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T20:59:54.662 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:54.363+0000 7f2434259200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:59:54.663 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:54.456+0000 7f2434259200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:59:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f46900a3c90 msgr2=0x7f46900a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:59:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f46900a3c90 0x7f46900a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 msgr2=0x7f46a419da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 0x7f46a419da30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f469c00ef30 tx=0x7f469c00c3a0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 shutdown_connections 2024-04-25T20:59:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f46900a3c90 0x7f46900a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4108ab0 0x7f46a419da30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a4102ac0 0x7f46a4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 --2- 172.21.15.47:0/1443010188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46a4103480 0x7f46a41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.663+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 >> 172.21.15.47:0/1443010188 conn(0x7f46a4079bf0 msgr2=0x7f46a4100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.664+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 shutdown_connections 2024-04-25T20:59:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:54.664+0000 7f46aa4b2700 1 -- 172.21.15.47:0/1443010188 wait complete. 2024-04-25T20:59:54.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:54 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1443010188' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T20:59:54.859 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:54.641+0000 7f143c2cd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:59:54.859 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:54.858+0000 7f143c2cd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:59:54.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:54 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1443010188' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T20:59:54.970 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:54.661+0000 7f2434259200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:59:54.971 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:54.856+0000 7f2434259200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:59:55.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:54 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1443010188' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T20:59:55.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:54.625+0000 7f0e6418a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:59:55.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:54.819+0000 7f0e6418a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:59:55.110 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:55.064+0000 7f143c2cd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T20:59:55.338 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T20:59:55.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:55.339 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-25T20:59:55.354 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T20:59:55.354 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T20:59:55.354 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42047sec preferred_lft 42047sec 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-25T20:59:55.355 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-25T20:59:55.356 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T20:59:55.356 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T20:59:55.356 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-25T20:59:55.356 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T20:59:55.356 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:59:55.356 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-25T20:59:55.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-04-25T20:59:55.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-04-25T20:59:55.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-04-25T20:59:55.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T20:59:55.357 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-25T20:59:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:59:55.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:55.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:55.547 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T20:59:55.573 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-25T20:59:55.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:55.575 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-25T20:59:55.648 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-25T20:59:55.648 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T20:59:55.648 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T20:59:55.648 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T20:59:55.648 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T20:59:55.649 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:59:55.649 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-25T20:59:55.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T20:59:55.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T20:59:55.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T20:59:55.649 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-25T20:59:55.720 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:55.464+0000 7f2434259200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:59:55.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:59:55.782 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:55 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:55.418+0000 7f0e6418a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:59:55.832 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:55.711+0000 7f143c2cd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:59:55.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:55.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:55.927 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-25T20:59:55.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:55.943 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-25T20:59:55.988 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:55.890+0000 7f2434259200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:59:55.988 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:55.987+0000 7f2434259200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:59:55.999 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:59:55.999 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-25T20:59:55.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T20:59:55.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T20:59:55.999 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-25T20:59:56.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:59:56.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:56.120 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T20:59:56.121 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:59:56.121 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-25T20:59:56.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T20:59:56.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T20:59:56.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T20:59:56.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T20:59:56.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T20:59:56.121 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-25T20:59:56.137 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:55 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:55.839+0000 7f0e6418a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:59:56.137 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:55 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:55.935+0000 7f0e6418a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:59:56.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:59:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:56.243 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:59:56.243 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-25T20:59:56.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-25T20:59:56.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T20:59:56.243 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-25T20:59:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:59:56.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:56.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T20:59:56.322 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T20:59:56.351 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T20:59:56.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T20:59:56.351 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T20:59:56.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:56.175+0000 7f143c2cd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:59:56.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:56.287+0000 7f143c2cd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:59:56.375 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T20:59:56.375 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T20:59:56.375 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/5eababe3-75e2-499f-b960-7d667e1f9f2f 2024-04-25T20:59:56.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:56.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/5eababe3-75e2-499f-b960-7d667e1f9f2f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T20:59:56.469 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T20:59:56.469 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T20:59:56.470 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:56 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:56.191+0000 7f2434259200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:59:56.532 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:56.136+0000 7f0e6418a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:59:56.575 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T20:59:56.575 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T20:59:56.575 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.92:3300,172.21.15.186:3300" 2024-04-25T20:59:56.575 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with new device syntax: 0@0ca8de06-0345-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/5eababe3-75e2-499f-b960-7d667e1f9f2f 2024-04-25T20:59:56.576 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.47:3300/172.21.15.92:3300/172.21.15.186:3300" will pass to kernel 2024-04-25T20:59:56.576 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T20:59:56.577 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-25T20:59:56.577 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T20:59:56.577 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T20:59:56.577 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T20:59:56.579 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T20:59:56.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:59:56.579 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T20:59:56.605 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T20:59:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T20:59:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:56.708 INFO:teuthology.orchestra.run.smithi047.stdout:client24832 2024-04-25T20:59:56.708 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T20:59:56.709 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T20:59:56.726 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T20:59:56.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T20:59:56.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:56.824 INFO:teuthology.orchestra.run.smithi047.stdout:client24832 2024-04-25T20:59:56.825 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T20:59:56.861 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:56.509+0000 7f143c2cd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:59:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T20:59:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:56.882 INFO:teuthology.orchestra.run.smithi047.stdout:client24832 2024-04-25T20:59:56.883 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0ca8de06-0345-11ef-bc93-c7b262605968.client24832/status') 2024-04-25T20:59:56.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:59:56.983 INFO:teuthology.orchestra.run.smithi047.stdout:instance: client.24832 (3)172.21.15.47:0/284850877 2024-04-25T20:59:56.983 INFO:teuthology.orchestra.run.smithi047.stdout:blocklisted: false 2024-04-25T20:59:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records in 2024-04-25T20:59:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records out 2024-04-25T20:59:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:70 bytes copied, 2.3871e-05 s, 2.9 MB/s 2024-04-25T20:59:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:59:56.990 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T20:59:57.002 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T20:59:57.002 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T20:59:57.002 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T20:59:57.003 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:56.656+0000 7f0e6418a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:59:57.003 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:56.748+0000 7f0e6418a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:59:57.064 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:56 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:56.725+0000 7f2434259200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:59:57.064 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:56 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:56.818+0000 7f2434259200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:59:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 -- 172.21.15.47:0/1483313396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc102040 msgr2=0x7febcc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 --2- 172.21.15.47:0/1483313396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc102040 0x7febcc1024c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7febc0009900 tx=0x7febc002a410 comp rx=0 tx=0).stop 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 -- 172.21.15.47:0/1483313396 shutdown_connections 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 --2- 172.21.15.47:0/1483313396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc102a00 0x7febcc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 --2- 172.21.15.47:0/1483313396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc102040 0x7febcc1024c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 --2- 172.21.15.47:0/1483313396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febcc100e60 0x7febcc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 -- 172.21.15.47:0/1483313396 >> 172.21.15.47:0/1483313396 conn(0x7febcc0fc420 msgr2=0x7febcc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 -- 172.21.15.47:0/1483313396 shutdown_connections 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.107+0000 7febd253d640 1 -- 172.21.15.47:0/1483313396 wait complete. 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.108+0000 7febd253d640 1 Processor -- start 2024-04-25T20:59:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.108+0000 7febd253d640 1 -- start start 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.108+0000 7febd253d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc100e60 0x7febcc10b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.108+0000 7febd253d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febcc102040 0x7febcc109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.108+0000 7febd253d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc102a00 0x7febcc10a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.108+0000 7febd253d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10db30 con 0x7febcc100e60 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.108+0000 7febd253d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10d9b0 con 0x7febcc102040 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.108+0000 7febd253d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10dcb0 con 0x7febcc102a00 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc100e60 0x7febcc10b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc100e60 0x7febcc10b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35092/0 (socket says 172.21.15.47:35092) 2024-04-25T20:59:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 -- 172.21.15.47:0/358005020 learned_addr learned my addr 172.21.15.47:0/358005020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febd0ada640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc102a00 0x7febcc10a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcb7fe640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febcc102040 0x7febcc109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 -- 172.21.15.47:0/358005020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc102a00 msgr2=0x7febcc10a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc102a00 0x7febcc10a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 -- 172.21.15.47:0/358005020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febcc102040 msgr2=0x7febcc109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febcc102040 0x7febcc109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 -- 172.21.15.47:0/358005020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc10ac70 con 0x7febcc100e60 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febcbfff640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc100e60 0x7febcc10b860 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e9c0 tx=0x7febbc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febc97fa640 1 -- 172.21.15.47:0/358005020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc00cd80 con 0x7febcc100e60 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febcc1a2280 con 0x7febcc100e60 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febc97fa640 1 -- 172.21.15.47:0/358005020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febbc0035f0 con 0x7febcc100e60 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febc97fa640 1 -- 172.21.15.47:0/358005020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc010690 con 0x7febcc100e60 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.109+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febcc1a27f0 con 0x7febcc100e60 2024-04-25T20:59:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.110+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febcc108200 con 0x7febcc100e60 2024-04-25T20:59:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.111+0000 7febc97fa640 1 -- 172.21.15.47:0/358005020 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7febbc0037a0 con 0x7febcc100e60 2024-04-25T20:59:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.112+0000 7febc97fa640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7febb40a3ca0 0x7febb40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.112+0000 7febcb7fe640 1 -- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7febb40a3ca0 msgr2=0x7febb40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.112+0000 7febc97fa640 1 -- 172.21.15.47:0/358005020 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7febbc0d0830 con 0x7febcc100e60 2024-04-25T20:59:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.112+0000 7febcb7fe640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7febb40a3ca0 0x7febb40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:59:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.116+0000 7febc97fa640 1 -- 172.21.15.47:0/358005020 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febbc09d200 con 0x7febcc100e60 2024-04-25T20:59:57.282 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:56.992+0000 7f0e6418a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:59:57.282 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:57.126+0000 7f0e6418a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:59:57.282 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:57.221+0000 7f0e6418a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:59:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.313+0000 7febcb7fe640 1 -- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7febb40a3ca0 msgr2=0x7febb40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.313+0000 7febcb7fe640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7febb40a3ca0 0x7febb40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:59:57.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.344+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7febcc02d5c0 con 0x7febcc100e60 2024-04-25T20:59:57.360 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:57.077+0000 7f143c2cd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:59:57.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:57.180+0000 7f143c2cd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:59:57.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.395+0000 7febc97fa640 1 -- 172.21.15.47:0/358005020 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7febcc02d5c0 con 0x7febcc100e60 2024-04-25T20:59:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.398+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7febb40a3ca0 msgr2=0x7febb40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:59:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.398+0000 7febd253d640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7febb40a3ca0 0x7febb40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.398+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc100e60 msgr2=0x7febcc10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.398+0000 7febd253d640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc100e60 0x7febcc10b860 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e9c0 tx=0x7febbc00ee80 comp rx=0 tx=0).stop 2024-04-25T20:59:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.399+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 shutdown_connections 2024-04-25T20:59:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.399+0000 7febd253d640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7febb40a3ca0 0x7febb40a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.399+0000 7febd253d640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febcc102a00 0x7febcc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.399+0000 7febd253d640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febcc102040 0x7febcc109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.399+0000 7febd253d640 1 --2- 172.21.15.47:0/358005020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febcc100e60 0x7febcc10b860 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.399+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 >> 172.21.15.47:0/358005020 conn(0x7febcc0fc420 msgr2=0x7febcc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.399+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 shutdown_connections 2024-04-25T20:59:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.399+0000 7febd253d640 1 -- 172.21.15.47:0/358005020 wait complete. 2024-04-25T20:59:57.403 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:57.063+0000 7f2434259200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:59:57.403 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:57.198+0000 7f2434259200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:59:57.403 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:57.294+0000 7f2434259200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:59:57.411 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T20:59:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 -- 172.21.15.47:0/353340796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 msgr2=0x7f56b41024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 --2- 172.21.15.47:0/353340796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 0x7f56b41024c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009900 tx=0x7f56a002a410 comp rx=0 tx=0).stop 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 -- 172.21.15.47:0/353340796 shutdown_connections 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 --2- 172.21.15.47:0/353340796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b4102a00 0x7f56b4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 --2- 172.21.15.47:0/353340796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 0x7f56b41024c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 --2- 172.21.15.47:0/353340796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56b4100e60 0x7f56b4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 -- 172.21.15.47:0/353340796 >> 172.21.15.47:0/353340796 conn(0x7f56b40fc420 msgr2=0x7f56b40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 -- 172.21.15.47:0/353340796 shutdown_connections 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 -- 172.21.15.47:0/353340796 wait complete. 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 Processor -- start 2024-04-25T20:59:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 -- start start 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56b4100e60 0x7f56b4104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.518+0000 7f56bab29640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 0x7f56b41034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56bab29640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b4102a00 0x7f56b41039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56bab29640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f56b410c9f0 con 0x7f56b4102040 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56bab29640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56b410c870 con 0x7f56b4102a00 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56bab29640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f56b410cb70 con 0x7f56b4100e60 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 0x7f56b41034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b88c5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56b4100e60 0x7f56b4104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 0x7f56b41034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35102/0 (socket says 172.21.15.47:35102) 2024-04-25T20:59:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 -- 172.21.15.47:0/184899777 learned_addr learned my addr 172.21.15.47:0/184899777 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b90c6640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b4102a00 0x7f56b41039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56b4100e60 msgr2=0x7f56b4104e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56b4100e60 0x7f56b4104e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b4102a00 msgr2=0x7f56b41039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b4102a00 0x7f56b41039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 -- 172.21.15.47:0/184899777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b4104280 con 0x7f56b4102040 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b88c5640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56b4100e60 0x7f56b4104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b3fff640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 0x7f56b41034b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56a0005940 tx=0x7f56a0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a003f030 con 0x7f56b4102040 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a0003aa0 con 0x7f56b4102040 2024-04-25T20:59:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41a2210 con 0x7f56b4102040 2024-04-25T20:59:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.519+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0035070 con 0x7f56b4102040 2024-04-25T20:59:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.520+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41a2760 con 0x7f56b4102040 2024-04-25T20:59:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.521+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b41a2a80 con 0x7f56b4102040 2024-04-25T20:59:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.522+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f56a0042030 con 0x7f56b4102040 2024-04-25T20:59:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.522+0000 7f56b1ffb640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 0x7f56a40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.523+0000 7f56b88c5640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 msgr2=0x7f56a40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.523+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f56a00f1030 con 0x7f56b4102040 2024-04-25T20:59:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.523+0000 7f56b88c5640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 0x7f56a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:59:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.530+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56a00bd3b0 con 0x7f56b4102040 2024-04-25T20:59:57.638 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:57 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/358005020' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T20:59:57.638 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:57.336+0000 7f0e6418a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:59:57.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:57 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/358005020' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T20:59:57.694 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:57.442+0000 7f143c2cd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:59:57.694 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:57.591+0000 7f143c2cd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:59:57.694 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:57.693+0000 7f143c2cd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:59:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:57 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/358005020' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T20:59:57.720 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:57.413+0000 7f2434259200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:59:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.724+0000 7f56b88c5640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 msgr2=0x7f56a40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.724+0000 7f56b88c5640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 0x7f56a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:59:57.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:57.747+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 --> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f56b40fe660 con 0x7f56a40a3ca0 2024-04-25T20:59:58.032 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:57.637+0000 7f0e6418a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:59:58.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:57.817+0000 7f143c2cd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:59:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:58.125+0000 7f56b88c5640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 msgr2=0x7f56a40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:58.125+0000 7f56b88c5640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 0x7f56a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T20:59:58.220 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:57.719+0000 7f2434259200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:59:58.402 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:58.132+0000 7f143c2cd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:59:58.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:58 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/358005020' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T20:59:58.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:58 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:59:58.533 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:58.235+0000 7f0e6418a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:59:58.533 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 20:59:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y[33945]: 2024-04-25T20:59:58.453+0000 7f0e6418a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:59:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:58 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/358005020' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T20:59:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:58 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:59:58.721 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:58.324+0000 7f2434259200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:59:58.721 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 20:59:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z[34284]: 2024-04-25T20:59:58.540+0000 7f2434259200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:59:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:58 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/358005020' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T20:59:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:58 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:59:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:58.926+0000 7f56b88c5640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 msgr2=0x7f56a40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/253800519 2024-04-25T20:59:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:58.926+0000 7f56b88c5640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 0x7f56a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T20:59:59.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:59.017+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7f56a00b9a20 con 0x7f56b4102040 2024-04-25T20:59:59.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:59.017+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 msgr2=0x7f56a40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:59:59.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T20:59:59.017+0000 7f56b1ffb640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/253800519,v1:172.21.15.47:6801/253800519] conn(0x7f56a40a3ca0 0x7f56a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:58.773+0000 7f143c2cd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:59:59.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.004+0000 7f143c2cd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: Standby manager daemon y restarted 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: Standby manager daemon y started 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: Standby manager daemon z restarted 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: Standby manager daemon z started 2024-04-25T20:59:59.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: Active manager daemon x restarted 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: Activating manager daemon x 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: mgrmap e20: x(active, starting, since 0.0113002s), standbys: z, y 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:59:59.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:59.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: Manager daemon x is now available 2024-04-25T20:59:59.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:59.475 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:59:59.475 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 20:59:59 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:59:59.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: Standby manager daemon y restarted 2024-04-25T20:59:59.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: Standby manager daemon y started 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: Standby manager daemon z restarted 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: Standby manager daemon z started 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: Active manager daemon x restarted 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: Activating manager daemon x 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T20:59:59.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: mgrmap e20: x(active, starting, since 0.0113002s), standbys: z, y 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:59:59.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:59:59.535 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:59:59.536 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:59:59.536 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:59:59.536 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:59:59.536 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:59:59.536 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: Manager daemon x is now available 2024-04-25T20:59:59.536 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:59.536 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:59:59.536 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 20:59:59 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:59:59.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139b440700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139b440700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139b440700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139b440700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139b440700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.611 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139e446700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139e446700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139e446700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139e446700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T20:59:59.260+0000 7f139e446700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: Standby manager daemon y restarted 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: Standby manager daemon y started 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:59:59.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.92:0/3801939835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: Standby manager daemon z restarted 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: Standby manager daemon z started 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.? 172.21.15.186:0/2854987677' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: Active manager daemon x restarted 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: Activating manager daemon x 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: mgrmap e20: x(active, starting, since 0.0113002s), standbys: z, y 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:59:59.613 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:59.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:59:59.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T20:59:59.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:59.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:59:59.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 20:59:59 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T21:00:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.024+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f56a00c2400 con 0x7f56b4102040 2024-04-25T21:00:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.025+0000 7f56b1ffb640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f56a4052730 0x7f56a4052b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.025+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f56a405edd0 con 0x7f56a4052730 2024-04-25T21:00:00.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.028+0000 7f56b88c5640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f56a4052730 0x7f56a4052b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.031+0000 7f56b88c5640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f56a4052730 0x7f56a4052b10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f56a800a010 tx=0x7f56a8005f90 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:00.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.046+0000 7f56b1ffb640 1 -- 172.21.15.47:0/184899777 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f56a405edd0 con 0x7f56a4052730 2024-04-25T21:00:00.047 INFO:teuthology.orchestra.run.smithi047.stdout:{} 2024-04-25T21:00:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f56a4052730 msgr2=0x7f56a4052b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56bab29640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f56a4052730 0x7f56a4052b10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f56a800a010 tx=0x7f56a8005f90 comp rx=0 tx=0).stop 2024-04-25T21:00:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 msgr2=0x7f56b41034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56bab29640 1 --2- 172.21.15.47:0/184899777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56b4102040 0x7f56b41034b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56a0005940 tx=0x7f56a0031a50 comp rx=0 tx=0).stop 2024-04-25T21:00:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56b90c6640 1 -- 172.21.15.47:0/184899777 reap_dead start 2024-04-25T21:00:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 shutdown_connections 2024-04-25T21:00:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 >> 172.21.15.47:0/184899777 conn(0x7f56b40fc420 msgr2=0x7f56b40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 shutdown_connections 2024-04-25T21:00:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.048+0000 7f56bab29640 1 -- 172.21.15.47:0/184899777 wait complete. 2024-04-25T21:00:00.061 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T21:00:00.073 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T21:00:00.084 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T21:00:00.084 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T21:00:00.084 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T21:00:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.192+0000 7f605038e640 1 -- 172.21.15.47:0/2367830279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048107d50 msgr2=0x7f604810a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.192+0000 7f605038e640 1 --2- 172.21.15.47:0/2367830279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048107d50 0x7f604810a140 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6040009ca0 tx=0x7f604002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 -- 172.21.15.47:0/2367830279 shutdown_connections 2024-04-25T21:00:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 --2- 172.21.15.47:0/2367830279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048107d50 0x7f604810a140 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 --2- 172.21.15.47:0/2367830279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6048105420 0x7f6048107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 --2- 172.21.15.47:0/2367830279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6048102af0 0x7f6048104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 -- 172.21.15.47:0/2367830279 >> 172.21.15.47:0/2367830279 conn(0x7f60480fc420 msgr2=0x7f60480fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 -- 172.21.15.47:0/2367830279 shutdown_connections 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 -- 172.21.15.47:0/2367830279 wait complete. 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 Processor -- start 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 -- start start 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6048102af0 0x7f60481981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048105420 0x7f60481986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6048107d50 0x7f604819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f604810db00 con 0x7f6048105420 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f604810d980 con 0x7f6048107d50 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.193+0000 7f605038e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f604810dc80 con 0x7f6048102af0 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048105420 0x7f60481986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048105420 0x7f60481986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35280/0 (socket says 172.21.15.47:35280) 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 -- 172.21.15.47:0/1192480226 learned_addr learned my addr 172.21.15.47:0/1192480226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604e12a640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6048102af0 0x7f60481981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604e92b640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6048107d50 0x7f604819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 -- 172.21.15.47:0/1192480226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6048102af0 msgr2=0x7f60481981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6048102af0 0x7f60481981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 -- 172.21.15.47:0/1192480226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6048107d50 msgr2=0x7f604819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6048107d50 0x7f604819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 -- 172.21.15.47:0/1192480226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604819fd70 con 0x7f6048105420 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f604d929640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048105420 0x7f60481986e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f604400e9c0 tx=0x7f604400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f603f7fe640 1 -- 172.21.15.47:0/1192480226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604400cd80 con 0x7f6048105420 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f603f7fe640 1 -- 172.21.15.47:0/1192480226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60440054d0 con 0x7f6048105420 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60481a0060 con 0x7f6048105420 2024-04-25T21:00:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f603f7fe640 1 -- 172.21.15.47:0/1192480226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044010430 con 0x7f6048105420 2024-04-25T21:00:00.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.194+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60481a05b0 con 0x7f6048105420 2024-04-25T21:00:00.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.195+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60480fdf10 con 0x7f6048105420 2024-04-25T21:00:00.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.197+0000 7f603f7fe640 1 -- 172.21.15.47:0/1192480226 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f6044004fd0 con 0x7f6048105420 2024-04-25T21:00:00.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.197+0000 7f603f7fe640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f60340a3a60 0x7f60340a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.197+0000 7f604e12a640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f60340a3a60 0x7f60340a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.198+0000 7f603f7fe640 1 -- 172.21.15.47:0/1192480226 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f60440d05f0 con 0x7f6048105420 2024-04-25T21:00:00.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.198+0000 7f604e12a640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f60340a3a60 0x7f60340a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6038005fd0 tx=0x7f603800b390 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:00.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.202+0000 7f603f7fe640 1 -- 172.21.15.47:0/1192480226 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f604409d170 con 0x7f6048105420 2024-04-25T21:00:00.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:00:00.037+0000 7f1398bfb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:00:00.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:00:00.037+0000 7f1398bfb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:00:00.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:00:00.037+0000 7f1398bfb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:00:00.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:00:00.037+0000 7f1398bfb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:00:00.361 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:00:00.037+0000 7f1398bfb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:00:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.414+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f6048062390 con 0x7f60340a3a60 2024-04-25T21:00:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.482+0000 7f603f7fe640 1 -- 172.21.15.47:0/1192480226 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6048062390 con 0x7f60340a3a60 2024-04-25T21:00:00.484 INFO:teuthology.orchestra.run.smithi047.stdout:Schedule set for path / 2024-04-25T21:00:00.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f60340a3a60 msgr2=0x7f60340a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f60340a3a60 0x7f60340a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6038005fd0 tx=0x7f603800b390 comp rx=0 tx=0).stop 2024-04-25T21:00:00.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048105420 msgr2=0x7f60481986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048105420 0x7f60481986e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f604400e9c0 tx=0x7f604400ee80 comp rx=0 tx=0).stop 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 shutdown_connections 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f60340a3a60 0x7f60340a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6048107d50 0x7f604819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6048105420 0x7f60481986e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 --2- 172.21.15.47:0/1192480226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6048102af0 0x7f60481981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.484+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 >> 172.21.15.47:0/1192480226 conn(0x7f60480fc420 msgr2=0x7f6048103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.485+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 shutdown_connections 2024-04-25T21:00:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.485+0000 7f605038e640 1 -- 172.21.15.47:0/1192480226 wait complete. 2024-04-25T21:00:00.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T21:00:00.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 -- 172.21.15.47:0/3681633902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 msgr2=0x7fcad8102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 --2- 172.21.15.47:0/3681633902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 0x7fcad8102420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009900 tx=0x7fcac802a410 comp rx=0 tx=0).stop 2024-04-25T21:00:00.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 -- 172.21.15.47:0/3681633902 shutdown_connections 2024-04-25T21:00:00.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 --2- 172.21.15.47:0/3681633902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcad8103bb0 0x7fcad810a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 --2- 172.21.15.47:0/3681633902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad81031f0 0x7fcad8103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 --2- 172.21.15.47:0/3681633902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 0x7fcad8102420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 -- 172.21.15.47:0/3681633902 >> 172.21.15.47:0/3681633902 conn(0x7fcad80fd570 msgr2=0x7fcad80ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:00.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 -- 172.21.15.47:0/3681633902 shutdown_connections 2024-04-25T21:00:00.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.612+0000 7fcadd210640 1 -- 172.21.15.47:0/3681633902 wait complete. 2024-04-25T21:00:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.613+0000 7fcadd210640 1 Processor -- start 2024-04-25T21:00:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.613+0000 7fcadd210640 1 -- start start 2024-04-25T21:00:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.613+0000 7fcadd210640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 0x7fcad81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.613+0000 7fcadd210640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcad81031f0 0x7fcad81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.613+0000 7fcadd210640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad8103bb0 0x7fcad819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcadd210640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcad810db00 con 0x7fcad8102000 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcadd210640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcad810d980 con 0x7fcad8103bb0 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcadd210640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcad810dc80 con 0x7fcad81031f0 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 0x7fcad81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad8103bb0 0x7fcad819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcad81031f0 0x7fcad81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 0x7fcad81981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35296/0 (socket says 172.21.15.47:35296) 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 -- 172.21.15.47:0/300682911 learned_addr learned my addr 172.21.15.47:0/300682911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 -- 172.21.15.47:0/300682911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcad81031f0 msgr2=0x7fcad81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcad81031f0 0x7fcad81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 -- 172.21.15.47:0/300682911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad8103bb0 msgr2=0x7fcad819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad8103bb0 0x7fcad819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 -- 172.21.15.47:0/300682911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad819fd70 con 0x7fcad8102000 2024-04-25T21:00:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad759e640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad8103bb0 0x7fcad819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.614+0000 7fcad6d9d640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 0x7fcad81981a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcac8006010 tx=0x7fcac80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.615+0000 7fcabffff640 1 -- 172.21.15.47:0/300682911 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac803f030 con 0x7fcad8102000 2024-04-25T21:00:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.615+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad81a0000 con 0x7fcad8102000 2024-04-25T21:00:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.615+0000 7fcabffff640 1 -- 172.21.15.47:0/300682911 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac8003cb0 con 0x7fcad8102000 2024-04-25T21:00:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.615+0000 7fcabffff640 1 -- 172.21.15.47:0/300682911 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8035070 con 0x7fcad8102000 2024-04-25T21:00:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.615+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad81a04f0 con 0x7fcad8102000 2024-04-25T21:00:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.616+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad8109490 con 0x7fcad8102000 2024-04-25T21:00:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.617+0000 7fcabffff640 1 -- 172.21.15.47:0/300682911 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fcac8042030 con 0x7fcad8102000 2024-04-25T21:00:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.617+0000 7fcabffff640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcac00a3a60 0x7fcac00a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.617+0000 7fcabffff640 1 -- 172.21.15.47:0/300682911 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fcac80f06c0 con 0x7fcad8102000 2024-04-25T21:00:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.618+0000 7fcad659c640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcac00a3a60 0x7fcac00a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.618+0000 7fcad659c640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcac00a3a60 0x7fcac00a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcad8199660 tx=0x7fcacc00b390 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.622+0000 7fcabffff640 1 -- 172.21.15.47:0/300682911 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcac80bcc30 con 0x7fcad8102000 2024-04-25T21:00:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.834+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fcad80624f0 con 0x7fcac00a3a60 2024-04-25T21:00:00.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.892+0000 7fcabffff640 1 -- 172.21.15.47:0/300682911 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fcad80624f0 con 0x7fcac00a3a60 2024-04-25T21:00:00.894 INFO:teuthology.orchestra.run.smithi047.stdout:Retention added to path / 2024-04-25T21:00:00.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.894+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcac00a3a60 msgr2=0x7fcac00a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.894+0000 7fcadd210640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcac00a3a60 0x7fcac00a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcad8199660 tx=0x7fcacc00b390 comp rx=0 tx=0).stop 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.894+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 msgr2=0x7fcad81981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.894+0000 7fcadd210640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 0x7fcad81981a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcac8006010 tx=0x7fcac80042f0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.894+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 shutdown_connections 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.894+0000 7fcadd210640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcac00a3a60 0x7fcac00a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.894+0000 7fcadd210640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad8103bb0 0x7fcad819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.895+0000 7fcadd210640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcad81031f0 0x7fcad81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.895+0000 7fcadd210640 1 --2- 172.21.15.47:0/300682911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad8102000 0x7fcad81981a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.895+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 >> 172.21.15.47:0/300682911 conn(0x7fcad80fd570 msgr2=0x7fcad8106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.895+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 shutdown_connections 2024-04-25T21:00:00.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:00.895+0000 7fcadd210640 1 -- 172.21.15.47:0/300682911 wait complete. 2024-04-25T21:00:00.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T21:00:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.013+0000 7ffb0f810640 1 -- 172.21.15.47:0/2475719209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08107d40 msgr2=0x7ffb0810a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.013+0000 7ffb0f810640 1 --2- 172.21.15.47:0/2475719209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08107d40 0x7ffb0810a190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009ca0 tx=0x7ffb0002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.013+0000 7ffb0f810640 1 -- 172.21.15.47:0/2475719209 shutdown_connections 2024-04-25T21:00:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.013+0000 7ffb0f810640 1 --2- 172.21.15.47:0/2475719209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08107d40 0x7ffb0810a190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.013+0000 7ffb0f810640 1 --2- 172.21.15.47:0/2475719209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08105410 0x7ffb08107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 --2- 172.21.15.47:0/2475719209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08102ae0 0x7ffb08104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 -- 172.21.15.47:0/2475719209 >> 172.21.15.47:0/2475719209 conn(0x7ffb080fc4d0 msgr2=0x7ffb080fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 -- 172.21.15.47:0/2475719209 shutdown_connections 2024-04-25T21:00:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 -- 172.21.15.47:0/2475719209 wait complete. 2024-04-25T21:00:01.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 Processor -- start 2024-04-25T21:00:01.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 -- start start 2024-04-25T21:00:01.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08102ae0 0x7ffb0819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08105410 0x7ffb0819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08107d40 0x7ffb081a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0810db30 con 0x7ffb08105410 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0810d9b0 con 0x7ffb08102ae0 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.014+0000 7ffb0f810640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0810dcb0 con 0x7ffb08107d40 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08105410 0x7ffb0819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08105410 0x7ffb0819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35310/0 (socket says 172.21.15.47:35310) 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 -- 172.21.15.47:0/674442664 learned_addr learned my addr 172.21.15.47:0/674442664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0d5ac640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08102ae0 0x7ffb0819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0ddad640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08107d40 0x7ffb081a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 -- 172.21.15.47:0/674442664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08107d40 msgr2=0x7ffb081a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08107d40 0x7ffb081a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 -- 172.21.15.47:0/674442664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08102ae0 msgr2=0x7ffb0819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08102ae0 0x7ffb0819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 -- 172.21.15.47:0/674442664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb081a1f00 con 0x7ffb08105410 2024-04-25T21:00:01.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.015+0000 7ffb0cdab640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08105410 0x7ffb0819a820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400e9c0 tx=0x7ffb0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:01.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.016+0000 7ffafa7fc640 1 -- 172.21.15.47:0/674442664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400cd80 con 0x7ffb08105410 2024-04-25T21:00:01.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.016+0000 7ffafa7fc640 1 -- 172.21.15.47:0/674442664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb040054d0 con 0x7ffb08105410 2024-04-25T21:00:01.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.016+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb081a21f0 con 0x7ffb08105410 2024-04-25T21:00:01.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.016+0000 7ffafa7fc640 1 -- 172.21.15.47:0/674442664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04010430 con 0x7ffb08105410 2024-04-25T21:00:01.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.016+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb081a2790 con 0x7ffb08105410 2024-04-25T21:00:01.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.017+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb08109830 con 0x7ffb08105410 2024-04-25T21:00:01.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.018+0000 7ffafa7fc640 1 -- 172.21.15.47:0/674442664 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7ffb04004fd0 con 0x7ffb08105410 2024-04-25T21:00:01.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.019+0000 7ffafa7fc640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ffaf40a3ae0 0x7ffaf40a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.019+0000 7ffafa7fc640 1 -- 172.21.15.47:0/674442664 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ffb040d0450 con 0x7ffb08105410 2024-04-25T21:00:01.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.019+0000 7ffb0d5ac640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ffaf40a3ae0 0x7ffaf40a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.020+0000 7ffb0d5ac640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ffaf40a3ae0 0x7ffaf40a5fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009bd0 tx=0x7ffafc005f90 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:01.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.024+0000 7ffafa7fc640 1 -- 172.21.15.47:0/674442664 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb0409d010 con 0x7ffb08105410 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: [25/Apr/2024:20:59:59] ENGINE Bus STARTING 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: [25/Apr/2024:20:59:59] ENGINE Serving on https://172.21.15.47:7150 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: [25/Apr/2024:20:59:59] ENGINE Serving on http://172.21.15.47:8765 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: [25/Apr/2024:20:59:59] ENGINE Bus STARTED 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: mgrmap e21: x(active, since 1.01963s), standbys: z, y 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:01.112 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:00 smithi047 ceph-mon[29195]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:01.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: [25/Apr/2024:20:59:59] ENGINE Bus STARTING 2024-04-25T21:00:01.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: [25/Apr/2024:20:59:59] ENGINE Serving on https://172.21.15.47:7150 2024-04-25T21:00:01.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:01.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:01.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: [25/Apr/2024:20:59:59] ENGINE Serving on http://172.21.15.47:8765 2024-04-25T21:00:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: [25/Apr/2024:20:59:59] ENGINE Bus STARTED 2024-04-25T21:00:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: overall HEALTH_OK 2024-04-25T21:00:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: mgrmap e21: x(active, since 1.01963s), standbys: z, y 2024-04-25T21:00:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:01.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:00 smithi186 ceph-mon[32529]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:01.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.241+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ffb08062390 con 0x7ffaf40a3ae0 2024-04-25T21:00:01.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.248+0000 7ffafa7fc640 1 -- 172.21.15.47:0/674442664 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7ffb08062390 con 0x7ffaf40a3ae0 2024-04-25T21:00:01.250 INFO:teuthology.orchestra.run.smithi047.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T21:00:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T21:00:01.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ffaf40a3ae0 msgr2=0x7ffaf40a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ffaf40a3ae0 0x7ffaf40a5fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009bd0 tx=0x7ffafc005f90 comp rx=0 tx=0).stop 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08105410 msgr2=0x7ffb0819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08105410 0x7ffb0819a820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400e9c0 tx=0x7ffb0400ee80 comp rx=0 tx=0).stop 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 shutdown_connections 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08107d40 0x7ffb081a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb08105410 0x7ffb0819a820 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ffaf40a3ae0 0x7ffaf40a5fa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 --2- 172.21.15.47:0/674442664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08102ae0 0x7ffb0819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 >> 172.21.15.47:0/674442664 conn(0x7ffb080fc4d0 msgr2=0x7ffb08102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 shutdown_connections 2024-04-25T21:00:01.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.251+0000 7ffb0f810640 1 -- 172.21.15.47:0/674442664 wait complete. 2024-04-25T21:00:01.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T21:00:01.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: [25/Apr/2024:20:59:59] ENGINE Bus STARTING 2024-04-25T21:00:01.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: [25/Apr/2024:20:59:59] ENGINE Serving on https://172.21.15.47:7150 2024-04-25T21:00:01.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:01.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:01.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: [25/Apr/2024:20:59:59] ENGINE Serving on http://172.21.15.47:8765 2024-04-25T21:00:01.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: [25/Apr/2024:20:59:59] ENGINE Bus STARTED 2024-04-25T21:00:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: overall HEALTH_OK 2024-04-25T21:00:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: mgrmap e21: x(active, since 1.01963s), standbys: z, y 2024-04-25T21:00:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:01.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:00 smithi092 ceph-mon[32524]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:01.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 -- 172.21.15.47:0/814515672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0100e80 msgr2=0x7f57e01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/814515672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0100e80 0x7f57e01012a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009ca0 tx=0x7f57d002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 -- 172.21.15.47:0/814515672 shutdown_connections 2024-04-25T21:00:01.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/814515672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0102a20 0x7f57e0109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/814515672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57e0102060 0x7f57e01024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/814515672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0100e80 0x7f57e01012a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 -- 172.21.15.47:0/814515672 >> 172.21.15.47:0/814515672 conn(0x7f57e00fc460 msgr2=0x7f57e00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:01.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 -- 172.21.15.47:0/814515672 shutdown_connections 2024-04-25T21:00:01.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.369+0000 7f57e5c4e640 1 -- 172.21.15.47:0/814515672 wait complete. 2024-04-25T21:00:01.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57e5c4e640 1 Processor -- start 2024-04-25T21:00:01.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57e5c4e640 1 -- start start 2024-04-25T21:00:01.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57e5c4e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0100e80 0x7f57e01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57e5c4e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0102060 0x7f57e01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57e5c4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57e0102a20 0x7f57e019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57e5c4e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010db70 con 0x7f57e0102060 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57e5c4e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010d9f0 con 0x7f57e0102a20 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57e5c4e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010dcf0 con 0x7f57e0100e80 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57deffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0102060 0x7f57e01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57deffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0102060 0x7f57e01986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35340/0 (socket says 172.21.15.47:35340) 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57deffd640 1 -- 172.21.15.47:0/4191017955 learned_addr learned my addr 172.21.15.47:0/4191017955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57df7fe640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0100e80 0x7f57e01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.370+0000 7f57dffff640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57e0102a20 0x7f57e019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57deffd640 1 -- 172.21.15.47:0/4191017955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0100e80 msgr2=0x7f57e01981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57deffd640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0100e80 0x7f57e01981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57deffd640 1 -- 172.21.15.47:0/4191017955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57e0102a20 msgr2=0x7f57e019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57deffd640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57e0102a20 0x7f57e019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57deffd640 1 -- 172.21.15.47:0/4191017955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e019fdc0 con 0x7f57e0102060 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57deffd640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0102060 0x7f57e01986e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d950 tx=0x7f57d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57dcff9640 1 -- 172.21.15.47:0/4191017955 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4017030 con 0x7f57e0102060 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57dcff9640 1 -- 172.21.15.47:0/4191017955 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d4004020 con 0x7f57e0102060 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e01a00b0 con 0x7f57e0102060 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57dcff9640 1 -- 172.21.15.47:0/4191017955 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d40047a0 con 0x7f57e0102060 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.371+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e01a0600 con 0x7f57e0102060 2024-04-25T21:00:01.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.372+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e0108220 con 0x7f57e0102060 2024-04-25T21:00:01.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.374+0000 7f57dcff9640 1 -- 172.21.15.47:0/4191017955 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f57d4009a30 con 0x7f57e0102060 2024-04-25T21:00:01.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.375+0000 7f57dcff9640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f57c80a3a60 0x7f57c80a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.375+0000 7f57dcff9640 1 -- 172.21.15.47:0/4191017955 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f57d40d01d0 con 0x7f57e0102060 2024-04-25T21:00:01.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.375+0000 7f57df7fe640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f57c80a3a60 0x7f57c80a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.376+0000 7f57df7fe640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f57c80a3a60 0x7f57c80a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57d0000f80 tx=0x7f57d0005f30 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:01.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.379+0000 7f57dcff9640 1 -- 172.21.15.47:0/4191017955 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57d409cd50 con 0x7f57e0102060 2024-04-25T21:00:01.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.621+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f57e0062390 con 0x7f57c80a3a60 2024-04-25T21:00:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.646+0000 7f57dcff9640 1 -- 172.21.15.47:0/4191017955 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f57e0062390 con 0x7f57c80a3a60 2024-04-25T21:00:01.648 INFO:teuthology.orchestra.run.smithi047.stdout:/ 1m 6m3h 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f57c80a3a60 msgr2=0x7f57c80a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f57c80a3a60 0x7f57c80a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57d0000f80 tx=0x7f57d0005f30 comp rx=0 tx=0).stop 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0102060 msgr2=0x7f57e01986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0102060 0x7f57e01986e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d950 tx=0x7f57d400de10 comp rx=0 tx=0).stop 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 shutdown_connections 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f57c80a3a60 0x7f57c80a5f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57e0102a20 0x7f57e019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57e0102060 0x7f57e01986e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 --2- 172.21.15.47:0/4191017955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0100e80 0x7f57e01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 >> 172.21.15.47:0/4191017955 conn(0x7f57e00fc460 msgr2=0x7f57e01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 shutdown_connections 2024-04-25T21:00:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:01.649+0000 7f57e5c4e640 1 -- 172.21.15.47:0/4191017955 wait complete. 2024-04-25T21:00:01.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T21:00:01.689 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T21:00:01.699 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T21:00:01.755 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T21:00:01.755 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T21:00:01.758 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T21:00:01.770 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T21:00:01.770 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:00:02.041 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:02 smithi047 ceph-mon[29195]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T21:00:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:02 smithi186 ceph-mon[32529]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T21:00:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:02.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:02 smithi092 ceph-mon[32524]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T21:00:03.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: Unable to set osd_memory_target on smithi186 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-04-25T21:00:03.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:03 smithi092 ceph-mon[32524]: Unable to set osd_memory_target on smithi092 to 184781721: error parsing value: Value '184781721' is below minimum 939524096 2024-04-25T21:00:03.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: Unable to set osd_memory_target on smithi186 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-04-25T21:00:03.591 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:03 smithi047 ceph-mon[29195]: Unable to set osd_memory_target on smithi092 to 184781721: error parsing value: Value '184781721' is below minimum 939524096 2024-04-25T21:00:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: Unable to set osd_memory_target on smithi186 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-04-25T21:00:03.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:03 smithi186 ceph-mon[32529]: Unable to set osd_memory_target on smithi092 to 184781721: error parsing value: Value '184781721' is below minimum 939524096 2024-04-25T21:00:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 -- 172.21.15.47:0/4221326760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316c103950 msgr2=0x7f316c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 --2- 172.21.15.47:0/4221326760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316c103950 0x7f316c103d30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3168009990 tx=0x7f3168009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 -- 172.21.15.47:0/4221326760 shutdown_connections 2024-04-25T21:00:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 --2- 172.21.15.47:0/4221326760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316c103950 0x7f316c103d30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 --2- 172.21.15.47:0/4221326760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c101ad0 0x7f316c1117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 --2- 172.21.15.47:0/4221326760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c101030 0x7f316c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 -- 172.21.15.47:0/4221326760 >> 172.21.15.47:0/4221326760 conn(0x7f316c077fb0 msgr2=0x7f316c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 -- 172.21.15.47:0/4221326760 shutdown_connections 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.061+0000 7f31743a0700 1 -- 172.21.15.47:0/4221326760 wait complete. 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f31743a0700 1 Processor -- start 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f31743a0700 1 -- start start 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f31743a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c101030 0x7f316c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f31743a0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316c101ad0 0x7f316c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f31743a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c103950 0x7f316c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f31743a0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f316c114e10 con 0x7f316c101ad0 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f31743a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f316c114c90 con 0x7f316c101030 2024-04-25T21:00:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f31743a0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f316c114f90 con 0x7f316c103950 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f317213c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c101030 0x7f316c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f317213c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c101030 0x7f316c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:40286/0 (socket says 172.21.15.47:40286) 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f317293d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c103950 0x7f316c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f317193b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316c101ad0 0x7f316c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.062+0000 7f317213c700 1 -- 172.21.15.47:0/528787506 learned_addr learned my addr 172.21.15.47:0/528787506 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f317213c700 1 -- 172.21.15.47:0/528787506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c103950 msgr2=0x7f316c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f317213c700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c103950 0x7f316c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f317213c700 1 -- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316c101ad0 msgr2=0x7f316c19b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f317213c700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316c101ad0 0x7f316c19b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f317213c700 1 -- 172.21.15.47:0/528787506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c1a0310 con 0x7f316c101030 2024-04-25T21:00:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f317293d700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c103950 0x7f316c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f317213c700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c101030 0x7f316c19b300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3168000c00 tx=0x7f316800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f31637fe700 1 -- 172.21.15.47:0/528787506 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168027070 con 0x7f316c101030 2024-04-25T21:00:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f31637fe700 1 -- 172.21.15.47:0/528787506 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3168022470 con 0x7f316c101030 2024-04-25T21:00:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c1a05a0 con 0x7f316c101030 2024-04-25T21:00:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f31637fe700 1 -- 172.21.15.47:0/528787506 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168017650 con 0x7f316c101030 2024-04-25T21:00:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.063+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1a7e80 con 0x7f316c101030 2024-04-25T21:00:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.064+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c102040 con 0x7f316c101030 2024-04-25T21:00:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.066+0000 7f31637fe700 1 -- 172.21.15.47:0/528787506 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f316802a070 con 0x7f316c101030 2024-04-25T21:00:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.067+0000 7f31637fe700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f31580a3ed0 0x7f31580a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.067+0000 7f31637fe700 1 -- 172.21.15.47:0/528787506 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f31680d1760 con 0x7f316c101030 2024-04-25T21:00:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.067+0000 7f317193b700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f31580a3ed0 0x7f31580a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.067+0000 7f317193b700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f31580a3ed0 0x7f31580a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f316c1006a0 tx=0x7f315c009260 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.072+0000 7f31637fe700 1 -- 172.21.15.47:0/528787506 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3168013070 con 0x7f316c101030 2024-04-25T21:00:04.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.384+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f316c19c660 con 0x7f31580a3ed0 2024-04-25T21:00:04.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.392+0000 7f31637fe700 1 -- 172.21.15.47:0/528787506 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257743 (secure 0 0 0) 0x7f316c19c660 con 0x7f31580a3ed0 2024-04-25T21:00:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:00:04.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f31580a3ed0 msgr2=0x7f31580a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f31580a3ed0 0x7f31580a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f316c1006a0 tx=0x7f315c009260 comp rx=0 tx=0).stop 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c101030 msgr2=0x7f316c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c101030 0x7f316c19b300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3168000c00 tx=0x7f316800bfd0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 shutdown_connections 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f31580a3ed0 0x7f31580a6390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c103950 0x7f316c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c101030 0x7f316c19b300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 --2- 172.21.15.47:0/528787506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f316c101ad0 0x7f316c19b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 >> 172.21.15.47:0/528787506 conn(0x7f316c077fb0 msgr2=0x7f316c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 shutdown_connections 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:04.395+0000 7f31743a0700 1 -- 172.21.15.47:0/528787506 wait complete. 2024-04-25T21:00:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-25T21:00:04.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: Unable to set osd_memory_target on smithi047 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T21:00:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-25T21:00:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T21:00:04.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:04 smithi047 ceph-mon[29195]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T21:00:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T21:00:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: Unable to set osd_memory_target on smithi047 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T21:00:04.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:04 smithi186 ceph-mon[32529]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T21:00:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:00:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T21:00:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: Unable to set osd_memory_target on smithi047 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: Updating smithi092:/etc/ceph/ceph.conf 2024-04-25T21:00:04.783 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:04 smithi092 ceph-mon[32524]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-25T21:00:05.097 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T21:00:03.028919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187459,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1295,"num_read_kb":809,"num_write":476,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365124,"kb_used_data":15584,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124576700,"statfs":{"total":1151940427776,"available":1151566540800,"internally_reserved":0,"allocated":15958016,"data_stored":9464083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.992921"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113432+0000","last_change":"2024-04-25T20:56:51.283722+0000","last_active":"2024-04-25T20:59:59.113432+0000","last_peered":"2024-04-25T20:59:59.113432+0000","last_clean":"2024-04-25T20:59:59.113432+0000","last_became_active":"2024-04-25T20:56:51.283359+0000","last_became_peered":"2024-04-25T20:56:51.283359+0000","last_unstale":"2024-04-25T20:59:59.113432+0000","last_undegraded":"2024-04-25T20:59:59.113432+0000","last_fullsized":"2024-04-25T20:59:59.113432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:47.314687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514767+0000","last_change":"2024-04-25T20:56:51.273199+0000","last_active":"2024-04-25T20:59:59.514767+0000","last_peered":"2024-04-25T20:59:59.514767+0000","last_clean":"2024-04-25T20:59:59.514767+0000","last_became_active":"2024-04-25T20:56:51.272838+0000","last_became_peered":"2024-04-25T20:56:51.272838+0000","last_unstale":"2024-04-25T20:59:59.514767+0000","last_undegraded":"2024-04-25T20:59:59.514767+0000","last_fullsized":"2024-04-25T20:59:59.514767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:58.033315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514639+0000","last_change":"2024-04-25T20:56:51.281461+0000","last_active":"2024-04-25T20:59:59.514639+0000","last_peered":"2024-04-25T20:59:59.514639+0000","last_clean":"2024-04-25T20:59:59.514639+0000","last_became_active":"2024-04-25T20:56:51.281141+0000","last_became_peered":"2024-04-25T20:56:51.281141+0000","last_unstale":"2024-04-25T20:59:59.514639+0000","last_undegraded":"2024-04-25T20:59:59.514639+0000","last_fullsized":"2024-04-25T20:59:59.514639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041716+0000","last_change":"2024-04-25T20:56:51.283187+0000","last_active":"2024-04-25T20:59:59.041716+0000","last_peered":"2024-04-25T20:59:59.041716+0000","last_clean":"2024-04-25T20:59:59.041716+0000","last_became_active":"2024-04-25T20:56:51.282640+0000","last_became_peered":"2024-04-25T20:56:51.282640+0000","last_unstale":"2024-04-25T20:59:59.041716+0000","last_undegraded":"2024-04-25T20:59:59.041716+0000","last_fullsized":"2024-04-25T20:59:59.041716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:15.588694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016083+0000","last_change":"2024-04-25T20:56:51.273204+0000","last_active":"2024-04-25T20:59:59.016083+0000","last_peered":"2024-04-25T20:59:59.016083+0000","last_clean":"2024-04-25T20:59:59.016083+0000","last_became_active":"2024-04-25T20:56:51.272126+0000","last_became_peered":"2024-04-25T20:56:51.272126+0000","last_unstale":"2024-04-25T20:59:59.016083+0000","last_undegraded":"2024-04-25T20:59:59.016083+0000","last_fullsized":"2024-04-25T20:59:59.016083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:31.684294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016332+0000","last_change":"2024-04-25T20:56:51.270762+0000","last_active":"2024-04-25T20:59:59.016332+0000","last_peered":"2024-04-25T20:59:59.016332+0000","last_clean":"2024-04-25T20:59:59.016332+0000","last_became_active":"2024-04-25T20:56:51.270444+0000","last_became_peered":"2024-04-25T20:56:51.270444+0000","last_unstale":"2024-04-25T20:59:59.016332+0000","last_undegraded":"2024-04-25T20:59:59.016332+0000","last_fullsized":"2024-04-25T20:59:59.016332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:00.155417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549002+0000","last_change":"2024-04-25T20:56:51.276029+0000","last_active":"2024-04-25T20:59:59.549002+0000","last_peered":"2024-04-25T20:59:59.549002+0000","last_clean":"2024-04-25T20:59:59.549002+0000","last_became_active":"2024-04-25T20:56:51.275272+0000","last_became_peered":"2024-04-25T20:56:51.275272+0000","last_unstale":"2024-04-25T20:59:59.549002+0000","last_undegraded":"2024-04-25T20:59:59.549002+0000","last_fullsized":"2024-04-25T20:59:59.549002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.428444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041346+0000","last_change":"2024-04-25T20:56:51.273769+0000","last_active":"2024-04-25T20:59:59.041346+0000","last_peered":"2024-04-25T20:59:59.041346+0000","last_clean":"2024-04-25T20:59:59.041346+0000","last_became_active":"2024-04-25T20:56:51.273412+0000","last_became_peered":"2024-04-25T20:56:51.273412+0000","last_unstale":"2024-04-25T20:59:59.041346+0000","last_undegraded":"2024-04-25T20:59:59.041346+0000","last_fullsized":"2024-04-25T20:59:59.041346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.324332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266500+0000","last_change":"2024-04-25T20:56:51.277097+0000","last_active":"2024-04-25T20:59:59.266500+0000","last_peered":"2024-04-25T20:59:59.266500+0000","last_clean":"2024-04-25T20:59:59.266500+0000","last_became_active":"2024-04-25T20:56:51.276768+0000","last_became_peered":"2024-04-25T20:56:51.276768+0000","last_unstale":"2024-04-25T20:59:59.266500+0000","last_undegraded":"2024-04-25T20:59:59.266500+0000","last_fullsized":"2024-04-25T20:59:59.266500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:23.276470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016837+0000","last_change":"2024-04-25T20:56:51.279046+0000","last_active":"2024-04-25T20:59:59.016837+0000","last_peered":"2024-04-25T20:59:59.016837+0000","last_clean":"2024-04-25T20:59:59.016837+0000","last_became_active":"2024-04-25T20:56:51.278701+0000","last_became_peered":"2024-04-25T20:56:51.278701+0000","last_unstale":"2024-04-25T20:59:59.016837+0000","last_undegraded":"2024-04-25T20:59:59.016837+0000","last_fullsized":"2024-04-25T20:59:59.016837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.370482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549375+0000","last_change":"2024-04-25T20:56:51.271593+0000","last_active":"2024-04-25T20:59:59.549375+0000","last_peered":"2024-04-25T20:59:59.549375+0000","last_clean":"2024-04-25T20:59:59.549375+0000","last_became_active":"2024-04-25T20:56:51.271224+0000","last_became_peered":"2024-04-25T20:56:51.271224+0000","last_unstale":"2024-04-25T20:59:59.549375+0000","last_undegraded":"2024-04-25T20:59:59.549375+0000","last_fullsized":"2024-04-25T20:59:59.549375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:46.847929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041553+0000","last_change":"2024-04-25T20:56:51.278281+0000","last_active":"2024-04-25T20:59:59.041553+0000","last_peered":"2024-04-25T20:59:59.041553+0000","last_clean":"2024-04-25T20:59:59.041553+0000","last_became_active":"2024-04-25T20:56:51.277947+0000","last_became_peered":"2024-04-25T20:56:51.277947+0000","last_unstale":"2024-04-25T20:59:59.041553+0000","last_undegraded":"2024-04-25T20:59:59.041553+0000","last_fullsized":"2024-04-25T20:59:59.041553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:02.509296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039058+0000","last_change":"2024-04-25T20:56:51.273784+0000","last_active":"2024-04-25T20:59:59.039058+0000","last_peered":"2024-04-25T20:59:59.039058+0000","last_clean":"2024-04-25T20:59:59.039058+0000","last_became_active":"2024-04-25T20:56:51.273489+0000","last_became_peered":"2024-04-25T20:56:51.273489+0000","last_unstale":"2024-04-25T20:59:59.039058+0000","last_undegraded":"2024-04-25T20:59:59.039058+0000","last_fullsized":"2024-04-25T20:59:59.039058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:00.765186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514418+0000","last_change":"2024-04-25T20:56:51.269829+0000","last_active":"2024-04-25T20:59:59.514418+0000","last_peered":"2024-04-25T20:59:59.514418+0000","last_clean":"2024-04-25T20:59:59.514418+0000","last_became_active":"2024-04-25T20:56:51.269446+0000","last_became_peered":"2024-04-25T20:56:51.269446+0000","last_unstale":"2024-04-25T20:59:59.514418+0000","last_undegraded":"2024-04-25T20:59:59.514418+0000","last_fullsized":"2024-04-25T20:59:59.514418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:12.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514335+0000","last_change":"2024-04-25T20:56:51.278117+0000","last_active":"2024-04-25T20:59:59.514335+0000","last_peered":"2024-04-25T20:59:59.514335+0000","last_clean":"2024-04-25T20:59:59.514335+0000","last_became_active":"2024-04-25T20:56:51.277596+0000","last_became_peered":"2024-04-25T20:56:51.277596+0000","last_unstale":"2024-04-25T20:59:59.514335+0000","last_undegraded":"2024-04-25T20:59:59.514335+0000","last_fullsized":"2024-04-25T20:59:59.514335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:28.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265776+0000","last_change":"2024-04-25T20:56:51.281575+0000","last_active":"2024-04-25T20:59:59.265776+0000","last_peered":"2024-04-25T20:59:59.265776+0000","last_clean":"2024-04-25T20:59:59.265776+0000","last_became_active":"2024-04-25T20:56:51.281217+0000","last_became_peered":"2024-04-25T20:56:51.281217+0000","last_unstale":"2024-04-25T20:59:59.265776+0000","last_undegraded":"2024-04-25T20:59:59.265776+0000","last_fullsized":"2024-04-25T20:59:59.265776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:05.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113547+0000","last_change":"2024-04-25T20:56:51.284608+0000","last_active":"2024-04-25T20:59:59.113547+0000","last_peered":"2024-04-25T20:59:59.113547+0000","last_clean":"2024-04-25T20:59:59.113547+0000","last_became_active":"2024-04-25T20:56:51.284257+0000","last_became_peered":"2024-04-25T20:56:51.284257+0000","last_unstale":"2024-04-25T20:59:59.113547+0000","last_undegraded":"2024-04-25T20:59:59.113547+0000","last_fullsized":"2024-04-25T20:59:59.113547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:44:59.184192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013207+0000","last_change":"2024-04-25T20:56:51.276770+0000","last_active":"2024-04-25T20:59:59.013207+0000","last_peered":"2024-04-25T20:59:59.013207+0000","last_clean":"2024-04-25T20:59:59.013207+0000","last_became_active":"2024-04-25T20:56:51.276431+0000","last_became_peered":"2024-04-25T20:56:51.276431+0000","last_unstale":"2024-04-25T20:59:59.013207+0000","last_undegraded":"2024-04-25T20:59:59.013207+0000","last_fullsized":"2024-04-25T20:59:59.013207+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:14:25.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265999+0000","last_change":"2024-04-25T20:56:51.283728+0000","last_active":"2024-04-25T20:59:59.265999+0000","last_peered":"2024-04-25T20:59:59.265999+0000","last_clean":"2024-04-25T20:59:59.265999+0000","last_became_active":"2024-04-25T20:56:51.283370+0000","last_became_peered":"2024-04-25T20:56:51.283370+0000","last_unstale":"2024-04-25T20:59:59.265999+0000","last_undegraded":"2024-04-25T20:59:59.265999+0000","last_fullsized":"2024-04-25T20:59:59.265999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:18.042769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012950+0000","last_change":"2024-04-25T20:56:48.259737+0000","last_active":"2024-04-25T20:59:59.012950+0000","last_peered":"2024-04-25T20:59:59.012950+0000","last_clean":"2024-04-25T20:59:59.012950+0000","last_became_active":"2024-04-25T20:56:48.259442+0000","last_became_peered":"2024-04-25T20:56:48.259442+0000","last_unstale":"2024-04-25T20:59:59.012950+0000","last_undegraded":"2024-04-25T20:59:59.012950+0000","last_fullsized":"2024-04-25T20:59:59.012950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:23.191536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042189+0000","last_change":"2024-04-25T20:56:51.274510+0000","last_active":"2024-04-25T20:59:59.042189+0000","last_peered":"2024-04-25T20:59:59.042189+0000","last_clean":"2024-04-25T20:59:59.042189+0000","last_became_active":"2024-04-25T20:56:51.274137+0000","last_became_peered":"2024-04-25T20:56:51.274137+0000","last_unstale":"2024-04-25T20:59:59.042189+0000","last_undegraded":"2024-04-25T20:59:59.042189+0000","last_fullsized":"2024-04-25T20:59:59.042189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:05.903617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042405+0000","last_change":"2024-04-25T20:56:48.947850+0000","last_active":"2024-04-25T20:59:59.042405+0000","last_peered":"2024-04-25T20:59:59.042405+0000","last_clean":"2024-04-25T20:59:59.042405+0000","last_became_active":"2024-04-25T20:56:48.946160+0000","last_became_peered":"2024-04-25T20:56:48.946160+0000","last_unstale":"2024-04-25T20:59:59.042405+0000","last_undegraded":"2024-04-25T20:59:59.042405+0000","last_fullsized":"2024-04-25T20:59:59.042405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:32.692390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.515020+0000","last_change":"2024-04-25T20:56:51.280478+0000","last_active":"2024-04-25T20:59:59.515020+0000","last_peered":"2024-04-25T20:59:59.515020+0000","last_clean":"2024-04-25T20:59:59.515020+0000","last_became_active":"2024-04-25T20:56:51.280173+0000","last_became_peered":"2024-04-25T20:56:51.280173+0000","last_unstale":"2024-04-25T20:59:59.515020+0000","last_undegraded":"2024-04-25T20:59:59.515020+0000","last_fullsized":"2024-04-25T20:59:59.515020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:48.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016506+0000","last_change":"2024-04-25T20:57:47.535264+0000","last_active":"2024-04-25T20:59:59.016506+0000","last_peered":"2024-04-25T20:59:59.016506+0000","last_clean":"2024-04-25T20:59:59.016506+0000","last_became_active":"2024-04-25T20:57:47.531152+0000","last_became_peered":"2024-04-25T20:57:47.531152+0000","last_unstale":"2024-04-25T20:59:59.016506+0000","last_undegraded":"2024-04-25T20:59:59.016506+0000","last_fullsized":"2024-04-25T20:59:59.016506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:19.085455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265973+0000","last_change":"2024-04-25T20:56:51.278383+0000","last_active":"2024-04-25T20:59:59.265973+0000","last_peered":"2024-04-25T20:59:59.265973+0000","last_clean":"2024-04-25T20:59:59.265973+0000","last_became_active":"2024-04-25T20:56:51.277603+0000","last_became_peered":"2024-04-25T20:56:51.277603+0000","last_unstale":"2024-04-25T20:59:59.265973+0000","last_undegraded":"2024-04-25T20:59:59.265973+0000","last_fullsized":"2024-04-25T20:59:59.265973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:12.197940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016420+0000","last_change":"2024-04-25T20:56:48.258869+0000","last_active":"2024-04-25T20:59:59.016420+0000","last_peered":"2024-04-25T20:59:59.016420+0000","last_clean":"2024-04-25T20:59:59.016420+0000","last_became_active":"2024-04-25T20:56:48.258414+0000","last_became_peered":"2024-04-25T20:56:48.258414+0000","last_unstale":"2024-04-25T20:59:59.016420+0000","last_undegraded":"2024-04-25T20:59:59.016420+0000","last_fullsized":"2024-04-25T20:59:59.016420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:09.889936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013004+0000","last_change":"2024-04-25T20:57:48.300637+0000","last_active":"2024-04-25T20:59:59.013004+0000","last_peered":"2024-04-25T20:59:59.013004+0000","last_clean":"2024-04-25T20:59:59.013004+0000","last_became_active":"2024-04-25T20:57:48.300285+0000","last_became_peered":"2024-04-25T20:57:48.300285+0000","last_unstale":"2024-04-25T20:59:59.013004+0000","last_undegraded":"2024-04-25T20:59:59.013004+0000","last_fullsized":"2024-04-25T20:59:59.013004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:31.810671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":211,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119667+0000","last_change":"2024-04-25T20:56:48.262543+0000","last_active":"2024-04-25T20:59:59.119667+0000","last_peered":"2024-04-25T20:59:59.119667+0000","last_clean":"2024-04-25T20:59:59.119667+0000","last_became_active":"2024-04-25T20:56:48.262086+0000","last_became_peered":"2024-04-25T20:56:48.262086+0000","last_unstale":"2024-04-25T20:59:59.119667+0000","last_undegraded":"2024-04-25T20:59:59.119667+0000","last_fullsized":"2024-04-25T20:59:59.119667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:43.722131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015962+0000","last_change":"2024-04-25T20:56:51.273500+0000","last_active":"2024-04-25T20:59:59.015962+0000","last_peered":"2024-04-25T20:59:59.015962+0000","last_clean":"2024-04-25T20:59:59.015962+0000","last_became_active":"2024-04-25T20:56:51.272882+0000","last_became_peered":"2024-04-25T20:56:51.272882+0000","last_unstale":"2024-04-25T20:59:59.015962+0000","last_undegraded":"2024-04-25T20:59:59.015962+0000","last_fullsized":"2024-04-25T20:59:59.015962+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:05.682258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016218+0000","last_change":"2024-04-25T20:56:48.255225+0000","last_active":"2024-04-25T20:59:59.016218+0000","last_peered":"2024-04-25T20:59:59.016218+0000","last_clean":"2024-04-25T20:59:59.016218+0000","last_became_active":"2024-04-25T20:56:48.254644+0000","last_became_peered":"2024-04-25T20:56:48.254644+0000","last_unstale":"2024-04-25T20:59:59.016218+0000","last_undegraded":"2024-04-25T20:59:59.016218+0000","last_fullsized":"2024-04-25T20:59:59.016218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:35.724612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119382+0000","last_change":"2024-04-25T20:56:51.270709+0000","last_active":"2024-04-25T20:59:59.119382+0000","last_peered":"2024-04-25T20:59:59.119382+0000","last_clean":"2024-04-25T20:59:59.119382+0000","last_became_active":"2024-04-25T20:56:51.270295+0000","last_became_peered":"2024-04-25T20:56:51.270295+0000","last_unstale":"2024-04-25T20:59:59.119382+0000","last_undegraded":"2024-04-25T20:59:59.119382+0000","last_fullsized":"2024-04-25T20:59:59.119382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:41.998953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038784+0000","last_change":"2024-04-25T20:56:48.256498+0000","last_active":"2024-04-25T20:59:59.038784+0000","last_peered":"2024-04-25T20:59:59.038784+0000","last_clean":"2024-04-25T20:59:59.038784+0000","last_became_active":"2024-04-25T20:56:48.256140+0000","last_became_peered":"2024-04-25T20:56:48.256140+0000","last_unstale":"2024-04-25T20:59:59.038784+0000","last_undegraded":"2024-04-25T20:59:59.038784+0000","last_fullsized":"2024-04-25T20:59:59.038784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:41.458235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041426+0000","last_change":"2024-04-25T20:56:51.267575+0000","last_active":"2024-04-25T20:59:59.041426+0000","last_peered":"2024-04-25T20:59:59.041426+0000","last_clean":"2024-04-25T20:59:59.041426+0000","last_became_active":"2024-04-25T20:56:51.267265+0000","last_became_peered":"2024-04-25T20:56:51.267265+0000","last_unstale":"2024-04-25T20:59:59.041426+0000","last_undegraded":"2024-04-25T20:59:59.041426+0000","last_fullsized":"2024-04-25T20:59:59.041426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:30.760555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119218+0000","last_change":"2024-04-25T20:56:48.260747+0000","last_active":"2024-04-25T20:59:59.119218+0000","last_peered":"2024-04-25T20:59:59.119218+0000","last_clean":"2024-04-25T20:59:59.119218+0000","last_became_active":"2024-04-25T20:56:48.260339+0000","last_became_peered":"2024-04-25T20:56:48.260339+0000","last_unstale":"2024-04-25T20:59:59.119218+0000","last_undegraded":"2024-04-25T20:59:59.119218+0000","last_fullsized":"2024-04-25T20:59:59.119218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:19.769631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012748+0000","last_change":"2024-04-25T20:57:47.795125+0000","last_active":"2024-04-25T20:59:59.012748+0000","last_peered":"2024-04-25T20:59:59.012748+0000","last_clean":"2024-04-25T20:59:59.012748+0000","last_became_active":"2024-04-25T20:57:47.794642+0000","last_became_peered":"2024-04-25T20:57:47.794642+0000","last_unstale":"2024-04-25T20:59:59.012748+0000","last_undegraded":"2024-04-25T20:59:59.012748+0000","last_fullsized":"2024-04-25T20:59:59.012748+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:15.443731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":205,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016821+0000","last_change":"2024-04-25T20:56:48.256113+0000","last_active":"2024-04-25T20:59:59.016821+0000","last_peered":"2024-04-25T20:59:59.016821+0000","last_clean":"2024-04-25T20:59:59.016821+0000","last_became_active":"2024-04-25T20:56:48.255644+0000","last_became_peered":"2024-04-25T20:56:48.255644+0000","last_unstale":"2024-04-25T20:59:59.016821+0000","last_undegraded":"2024-04-25T20:59:59.016821+0000","last_fullsized":"2024-04-25T20:59:59.016821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:12.100099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266407+0000","last_change":"2024-04-25T20:56:51.276990+0000","last_active":"2024-04-25T20:59:59.266407+0000","last_peered":"2024-04-25T20:59:59.266407+0000","last_clean":"2024-04-25T20:59:59.266407+0000","last_became_active":"2024-04-25T20:56:51.276638+0000","last_became_peered":"2024-04-25T20:56:51.276638+0000","last_unstale":"2024-04-25T20:59:59.266407+0000","last_undegraded":"2024-04-25T20:59:59.266407+0000","last_fullsized":"2024-04-25T20:59:59.266407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:06.645278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549502+0000","last_change":"2024-04-25T20:56:48.263216+0000","last_active":"2024-04-25T20:59:59.549502+0000","last_peered":"2024-04-25T20:59:59.549502+0000","last_clean":"2024-04-25T20:59:59.549502+0000","last_became_active":"2024-04-25T20:56:48.262854+0000","last_became_peered":"2024-04-25T20:56:48.262854+0000","last_unstale":"2024-04-25T20:59:59.549502+0000","last_undegraded":"2024-04-25T20:59:59.549502+0000","last_fullsized":"2024-04-25T20:59:59.549502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:00.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514957+0000","last_change":"2024-04-25T20:56:51.278479+0000","last_active":"2024-04-25T20:59:59.514957+0000","last_peered":"2024-04-25T20:59:59.514957+0000","last_clean":"2024-04-25T20:59:59.514957+0000","last_became_active":"2024-04-25T20:56:51.276285+0000","last_became_peered":"2024-04-25T20:56:51.276285+0000","last_unstale":"2024-04-25T20:59:59.514957+0000","last_undegraded":"2024-04-25T20:59:59.514957+0000","last_fullsized":"2024-04-25T20:59:59.514957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:14.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119578+0000","last_change":"2024-04-25T20:56:48.946808+0000","last_active":"2024-04-25T20:59:59.119578+0000","last_peered":"2024-04-25T20:59:59.119578+0000","last_clean":"2024-04-25T20:59:59.119578+0000","last_became_active":"2024-04-25T20:56:48.946421+0000","last_became_peered":"2024-04-25T20:56:48.946421+0000","last_unstale":"2024-04-25T20:59:59.119578+0000","last_undegraded":"2024-04-25T20:59:59.119578+0000","last_fullsized":"2024-04-25T20:59:59.119578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:47.763950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266116+0000","last_change":"2024-04-25T20:56:51.274761+0000","last_active":"2024-04-25T20:59:59.266116+0000","last_peered":"2024-04-25T20:59:59.266116+0000","last_clean":"2024-04-25T20:59:59.266116+0000","last_became_active":"2024-04-25T20:56:51.274452+0000","last_became_peered":"2024-04-25T20:56:51.274452+0000","last_unstale":"2024-04-25T20:59:59.266116+0000","last_undegraded":"2024-04-25T20:59:59.266116+0000","last_fullsized":"2024-04-25T20:59:59.266116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:32.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'36","reported_seq":423,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:00.011723+0000","last_change":"2024-04-25T20:56:48.258982+0000","last_active":"2024-04-25T21:00:00.011723+0000","last_peered":"2024-04-25T21:00:00.011723+0000","last_clean":"2024-04-25T21:00:00.011723+0000","last_became_active":"2024-04-25T20:56:48.258593+0000","last_became_peered":"2024-04-25T20:56:48.258593+0000","last_unstale":"2024-04-25T21:00:00.011723+0000","last_undegraded":"2024-04-25T21:00:00.011723+0000","last_fullsized":"2024-04-25T21:00:00.011723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:20.518202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":358,"num_read_kb":245,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016324+0000","last_change":"2024-04-25T20:56:51.276361+0000","last_active":"2024-04-25T20:59:59.016324+0000","last_peered":"2024-04-25T20:59:59.016324+0000","last_clean":"2024-04-25T20:59:59.016324+0000","last_became_active":"2024-04-25T20:56:51.276045+0000","last_became_peered":"2024-04-25T20:56:51.276045+0000","last_unstale":"2024-04-25T20:59:59.016324+0000","last_undegraded":"2024-04-25T20:59:59.016324+0000","last_fullsized":"2024-04-25T20:59:59.016324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:04.325068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266011+0000","last_change":"2024-04-25T20:56:48.252358+0000","last_active":"2024-04-25T20:59:59.266011+0000","last_peered":"2024-04-25T20:59:59.266011+0000","last_clean":"2024-04-25T20:59:59.266011+0000","last_became_active":"2024-04-25T20:56:48.252027+0000","last_became_peered":"2024-04-25T20:56:48.252027+0000","last_unstale":"2024-04-25T20:59:59.266011+0000","last_undegraded":"2024-04-25T20:59:59.266011+0000","last_fullsized":"2024-04-25T20:59:59.266011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:03.024487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038654+0000","last_change":"2024-04-25T20:56:51.279599+0000","last_active":"2024-04-25T20:59:59.038654+0000","last_peered":"2024-04-25T20:59:59.038654+0000","last_clean":"2024-04-25T20:59:59.038654+0000","last_became_active":"2024-04-25T20:56:51.279294+0000","last_became_peered":"2024-04-25T20:56:51.279294+0000","last_unstale":"2024-04-25T20:59:59.038654+0000","last_undegraded":"2024-04-25T20:59:59.038654+0000","last_fullsized":"2024-04-25T20:59:59.038654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119216+0000","last_change":"2024-04-25T20:56:48.261005+0000","last_active":"2024-04-25T20:59:59.119216+0000","last_peered":"2024-04-25T20:59:59.119216+0000","last_clean":"2024-04-25T20:59:59.119216+0000","last_became_active":"2024-04-25T20:56:48.260543+0000","last_became_peered":"2024-04-25T20:56:48.260543+0000","last_unstale":"2024-04-25T20:59:59.119216+0000","last_undegraded":"2024-04-25T20:59:59.119216+0000","last_fullsized":"2024-04-25T20:59:59.119216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:41:54.326036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549230+0000","last_change":"2024-04-25T20:56:51.271753+0000","last_active":"2024-04-25T20:59:59.549230+0000","last_peered":"2024-04-25T20:59:59.549230+0000","last_clean":"2024-04-25T20:59:59.549230+0000","last_became_active":"2024-04-25T20:56:51.271392+0000","last_became_peered":"2024-04-25T20:56:51.271392+0000","last_unstale":"2024-04-25T20:59:59.549230+0000","last_undegraded":"2024-04-25T20:59:59.549230+0000","last_fullsized":"2024-04-25T20:59:59.549230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:26.605647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":210,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041492+0000","last_change":"2024-04-25T20:56:48.260729+0000","last_active":"2024-04-25T20:59:59.041492+0000","last_peered":"2024-04-25T20:59:59.041492+0000","last_clean":"2024-04-25T20:59:59.041492+0000","last_became_active":"2024-04-25T20:56:48.260377+0000","last_became_peered":"2024-04-25T20:56:48.260377+0000","last_unstale":"2024-04-25T20:59:59.041492+0000","last_undegraded":"2024-04-25T20:59:59.041492+0000","last_fullsized":"2024-04-25T20:59:59.041492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:21.582266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.548948+0000","last_change":"2024-04-25T20:56:51.275691+0000","last_active":"2024-04-25T20:59:59.548948+0000","last_peered":"2024-04-25T20:59:59.548948+0000","last_clean":"2024-04-25T20:59:59.548948+0000","last_became_active":"2024-04-25T20:56:51.275220+0000","last_became_peered":"2024-04-25T20:56:51.275220+0000","last_unstale":"2024-04-25T20:59:59.548948+0000","last_undegraded":"2024-04-25T20:59:59.548948+0000","last_fullsized":"2024-04-25T20:59:59.548948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:24.367413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015925+0000","last_change":"2024-04-25T20:56:48.946465+0000","last_active":"2024-04-25T20:59:59.015925+0000","last_peered":"2024-04-25T20:59:59.015925+0000","last_clean":"2024-04-25T20:59:59.015925+0000","last_became_active":"2024-04-25T20:56:48.946093+0000","last_became_peered":"2024-04-25T20:56:48.946093+0000","last_unstale":"2024-04-25T20:59:59.015925+0000","last_undegraded":"2024-04-25T20:59:59.015925+0000","last_fullsized":"2024-04-25T20:59:59.015925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:29.502073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119264+0000","last_change":"2024-04-25T20:56:51.278464+0000","last_active":"2024-04-25T20:59:59.119264+0000","last_peered":"2024-04-25T20:59:59.119264+0000","last_clean":"2024-04-25T20:59:59.119264+0000","last_became_active":"2024-04-25T20:56:51.278119+0000","last_became_peered":"2024-04-25T20:56:51.278119+0000","last_unstale":"2024-04-25T20:59:59.119264+0000","last_undegraded":"2024-04-25T20:59:59.119264+0000","last_fullsized":"2024-04-25T20:59:59.119264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:57:37.224817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016353+0000","last_change":"2024-04-25T20:56:48.255123+0000","last_active":"2024-04-25T20:59:59.016353+0000","last_peered":"2024-04-25T20:59:59.016353+0000","last_clean":"2024-04-25T20:59:59.016353+0000","last_became_active":"2024-04-25T20:56:48.254680+0000","last_became_peered":"2024-04-25T20:56:48.254680+0000","last_unstale":"2024-04-25T20:59:59.016353+0000","last_undegraded":"2024-04-25T20:59:59.016353+0000","last_fullsized":"2024-04-25T20:59:59.016353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:24.089590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113399+0000","last_change":"2024-04-25T20:57:47.592950+0000","last_active":"2024-04-25T20:59:59.113399+0000","last_peered":"2024-04-25T20:59:59.113399+0000","last_clean":"2024-04-25T20:59:59.113399+0000","last_became_active":"2024-04-25T20:57:47.592486+0000","last_became_peered":"2024-04-25T20:57:47.592486+0000","last_unstale":"2024-04-25T20:59:59.113399+0000","last_undegraded":"2024-04-25T20:59:59.113399+0000","last_fullsized":"2024-04-25T20:59:59.113399+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:43.218061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012933+0000","last_change":"2024-04-25T20:56:48.947508+0000","last_active":"2024-04-25T20:59:59.012933+0000","last_peered":"2024-04-25T20:59:59.012933+0000","last_clean":"2024-04-25T20:59:59.012933+0000","last_became_active":"2024-04-25T20:56:48.946582+0000","last_became_peered":"2024-04-25T20:56:48.946582+0000","last_unstale":"2024-04-25T20:59:59.012933+0000","last_undegraded":"2024-04-25T20:59:59.012933+0000","last_fullsized":"2024-04-25T20:59:59.012933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:11.198667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016523+0000","last_change":"2024-04-25T20:56:51.273560+0000","last_active":"2024-04-25T20:59:59.016523+0000","last_peered":"2024-04-25T20:59:59.016523+0000","last_clean":"2024-04-25T20:59:59.016523+0000","last_became_active":"2024-04-25T20:56:51.273182+0000","last_became_peered":"2024-04-25T20:56:51.273182+0000","last_unstale":"2024-04-25T20:59:59.016523+0000","last_undegraded":"2024-04-25T20:59:59.016523+0000","last_fullsized":"2024-04-25T20:59:59.016523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:47.012771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549107+0000","last_change":"2024-04-25T20:56:48.260754+0000","last_active":"2024-04-25T20:59:59.549107+0000","last_peered":"2024-04-25T20:59:59.549107+0000","last_clean":"2024-04-25T20:59:59.549107+0000","last_became_active":"2024-04-25T20:56:48.260420+0000","last_became_peered":"2024-04-25T20:56:48.260420+0000","last_unstale":"2024-04-25T20:59:59.549107+0000","last_undegraded":"2024-04-25T20:59:59.549107+0000","last_fullsized":"2024-04-25T20:59:59.549107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:00.219023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016420+0000","last_change":"2024-04-25T20:56:51.271944+0000","last_active":"2024-04-25T20:59:59.016420+0000","last_peered":"2024-04-25T20:59:59.016420+0000","last_clean":"2024-04-25T20:59:59.016420+0000","last_became_active":"2024-04-25T20:56:51.270656+0000","last_became_peered":"2024-04-25T20:56:51.270656+0000","last_unstale":"2024-04-25T20:59:59.016420+0000","last_undegraded":"2024-04-25T20:59:59.016420+0000","last_fullsized":"2024-04-25T20:59:59.016420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:00.624888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012713+0000","last_change":"2024-04-25T20:57:47.526172+0000","last_active":"2024-04-25T20:59:59.012713+0000","last_peered":"2024-04-25T20:59:59.012713+0000","last_clean":"2024-04-25T20:59:59.012713+0000","last_became_active":"2024-04-25T20:57:47.525820+0000","last_became_peered":"2024-04-25T20:57:47.525820+0000","last_unstale":"2024-04-25T20:59:59.012713+0000","last_undegraded":"2024-04-25T20:59:59.012713+0000","last_fullsized":"2024-04-25T20:59:59.012713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:56.458959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":206,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038639+0000","last_change":"2024-04-25T20:56:48.946471+0000","last_active":"2024-04-25T20:59:59.038639+0000","last_peered":"2024-04-25T20:59:59.038639+0000","last_clean":"2024-04-25T20:59:59.038639+0000","last_became_active":"2024-04-25T20:56:48.946029+0000","last_became_peered":"2024-04-25T20:56:48.946029+0000","last_unstale":"2024-04-25T20:59:59.038639+0000","last_undegraded":"2024-04-25T20:59:59.038639+0000","last_fullsized":"2024-04-25T20:59:59.038639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:30:16.400157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":171,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013018+0000","last_change":"2024-04-25T20:57:48.300758+0000","last_active":"2024-04-25T20:59:59.013018+0000","last_peered":"2024-04-25T20:59:59.013018+0000","last_clean":"2024-04-25T20:59:59.013018+0000","last_became_active":"2024-04-25T20:57:48.300328+0000","last_became_peered":"2024-04-25T20:57:48.300328+0000","last_unstale":"2024-04-25T20:59:59.013018+0000","last_undegraded":"2024-04-25T20:59:59.013018+0000","last_fullsized":"2024-04-25T20:59:59.013018+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:31.931834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041515+0000","last_change":"2024-04-25T20:56:48.258479+0000","last_active":"2024-04-25T20:59:59.041515+0000","last_peered":"2024-04-25T20:59:59.041515+0000","last_clean":"2024-04-25T20:59:59.041515+0000","last_became_active":"2024-04-25T20:56:48.257951+0000","last_became_peered":"2024-04-25T20:56:48.257951+0000","last_unstale":"2024-04-25T20:59:59.041515+0000","last_undegraded":"2024-04-25T20:59:59.041515+0000","last_fullsized":"2024-04-25T20:59:59.041515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:13.812201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549048+0000","last_change":"2024-04-25T20:55:28.701357+0000","last_active":"2024-04-25T20:59:59.549048+0000","last_peered":"2024-04-25T20:59:59.549048+0000","last_clean":"2024-04-25T20:59:59.549048+0000","last_became_active":"2024-04-25T20:55:28.700906+0000","last_became_peered":"2024-04-25T20:55:28.700906+0000","last_unstale":"2024-04-25T20:59:59.549048+0000","last_undegraded":"2024-04-25T20:59:59.549048+0000","last_fullsized":"2024-04-25T20:59:59.549048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_clean_scrub_stamp":"2024-04-25T20:52:10.334323+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:37:35.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038561+0000","last_change":"2024-04-25T20:56:51.275249+0000","last_active":"2024-04-25T20:59:59.038561+0000","last_peered":"2024-04-25T20:59:59.038561+0000","last_clean":"2024-04-25T20:59:59.038561+0000","last_became_active":"2024-04-25T20:56:51.274938+0000","last_became_peered":"2024-04-25T20:56:51.274938+0000","last_unstale":"2024-04-25T20:59:59.038561+0000","last_undegraded":"2024-04-25T20:59:59.038561+0000","last_fullsized":"2024-04-25T20:59:59.038561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:53.933324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549168+0000","last_change":"2024-04-25T20:56:48.259357+0000","last_active":"2024-04-25T20:59:59.549168+0000","last_peered":"2024-04-25T20:59:59.549168+0000","last_clean":"2024-04-25T20:59:59.549168+0000","last_became_active":"2024-04-25T20:56:48.258840+0000","last_became_peered":"2024-04-25T20:56:48.258840+0000","last_unstale":"2024-04-25T20:59:59.549168+0000","last_undegraded":"2024-04-25T20:59:59.549168+0000","last_fullsized":"2024-04-25T20:59:59.549168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:30.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016354+0000","last_change":"2024-04-25T20:56:48.946498+0000","last_active":"2024-04-25T20:59:59.016354+0000","last_peered":"2024-04-25T20:59:59.016354+0000","last_clean":"2024-04-25T20:59:59.016354+0000","last_became_active":"2024-04-25T20:56:48.946124+0000","last_became_peered":"2024-04-25T20:56:48.946124+0000","last_unstale":"2024-04-25T20:59:59.016354+0000","last_undegraded":"2024-04-25T20:59:59.016354+0000","last_fullsized":"2024-04-25T20:59:59.016354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:08.545574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038986+0000","last_change":"2024-04-25T20:56:51.276093+0000","last_active":"2024-04-25T20:59:59.038986+0000","last_peered":"2024-04-25T20:59:59.038986+0000","last_clean":"2024-04-25T20:59:59.038986+0000","last_became_active":"2024-04-25T20:56:51.275771+0000","last_became_peered":"2024-04-25T20:56:51.275771+0000","last_unstale":"2024-04-25T20:59:59.038986+0000","last_undegraded":"2024-04-25T20:59:59.038986+0000","last_fullsized":"2024-04-25T20:59:59.038986+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:15.159208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'49","reported_seq":113,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.228897+0000","last_change":"2024-04-25T20:57:47.533338+0000","last_active":"2024-04-25T20:59:59.228897+0000","last_peered":"2024-04-25T20:59:59.228897+0000","last_clean":"2024-04-25T20:59:59.228897+0000","last_became_active":"2024-04-25T20:57:47.529539+0000","last_became_peered":"2024-04-25T20:57:47.529539+0000","last_unstale":"2024-04-25T20:59:59.228897+0000","last_undegraded":"2024-04-25T20:59:59.228897+0000","last_fullsized":"2024-04-25T20:59:59.228897+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:25.099128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042039+0000","last_change":"2024-04-25T20:56:51.275461+0000","last_active":"2024-04-25T20:59:59.042039+0000","last_peered":"2024-04-25T20:59:59.042039+0000","last_clean":"2024-04-25T20:59:59.042039+0000","last_became_active":"2024-04-25T20:56:51.275087+0000","last_became_peered":"2024-04-25T20:56:51.275087+0000","last_unstale":"2024-04-25T20:59:59.042039+0000","last_undegraded":"2024-04-25T20:59:59.042039+0000","last_fullsized":"2024-04-25T20:59:59.042039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:01:02.514490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041635+0000","last_change":"2024-04-25T20:56:49.031177+0000","last_active":"2024-04-25T20:59:59.041635+0000","last_peered":"2024-04-25T20:59:59.041635+0000","last_clean":"2024-04-25T20:59:59.041635+0000","last_became_active":"2024-04-25T20:56:49.030697+0000","last_became_peered":"2024-04-25T20:56:49.030697+0000","last_unstale":"2024-04-25T20:59:59.041635+0000","last_undegraded":"2024-04-25T20:59:59.041635+0000","last_fullsized":"2024-04-25T20:59:59.041635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:13.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514801+0000","last_change":"2024-04-25T20:57:47.795434+0000","last_active":"2024-04-25T20:59:59.514801+0000","last_peered":"2024-04-25T20:59:59.514801+0000","last_clean":"2024-04-25T20:59:59.514801+0000","last_became_active":"2024-04-25T20:57:47.795036+0000","last_became_peered":"2024-04-25T20:57:47.795036+0000","last_unstale":"2024-04-25T20:59:59.514801+0000","last_undegraded":"2024-04-25T20:59:59.514801+0000","last_fullsized":"2024-04-25T20:59:59.514801+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:48:56.749328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265934+0000","last_change":"2024-04-25T20:56:48.946689+0000","last_active":"2024-04-25T20:59:59.265934+0000","last_peered":"2024-04-25T20:59:59.265934+0000","last_clean":"2024-04-25T20:59:59.265934+0000","last_became_active":"2024-04-25T20:56:48.946229+0000","last_became_peered":"2024-04-25T20:56:48.946229+0000","last_unstale":"2024-04-25T20:59:59.265934+0000","last_undegraded":"2024-04-25T20:59:59.265934+0000","last_fullsized":"2024-04-25T20:59:59.265934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:04.120197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015968+0000","last_change":"2024-04-25T20:56:51.275657+0000","last_active":"2024-04-25T20:59:59.015968+0000","last_peered":"2024-04-25T20:59:59.015968+0000","last_clean":"2024-04-25T20:59:59.015968+0000","last_became_active":"2024-04-25T20:56:51.275237+0000","last_became_peered":"2024-04-25T20:56:51.275237+0000","last_unstale":"2024-04-25T20:59:59.015968+0000","last_undegraded":"2024-04-25T20:59:59.015968+0000","last_fullsized":"2024-04-25T20:59:59.015968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:39.778340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041385+0000","last_change":"2024-04-25T20:56:48.251882+0000","last_active":"2024-04-25T20:59:59.041385+0000","last_peered":"2024-04-25T20:59:59.041385+0000","last_clean":"2024-04-25T20:59:59.041385+0000","last_became_active":"2024-04-25T20:56:48.251571+0000","last_became_peered":"2024-04-25T20:56:48.251571+0000","last_unstale":"2024-04-25T20:59:59.041385+0000","last_undegraded":"2024-04-25T20:59:59.041385+0000","last_fullsized":"2024-04-25T20:59:59.041385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:51.769332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113156+0000","last_change":"2024-04-25T20:56:51.280648+0000","last_active":"2024-04-25T20:59:59.113156+0000","last_peered":"2024-04-25T20:59:59.113156+0000","last_clean":"2024-04-25T20:59:59.113156+0000","last_became_active":"2024-04-25T20:56:51.280232+0000","last_became_peered":"2024-04-25T20:56:51.280232+0000","last_unstale":"2024-04-25T20:59:59.113156+0000","last_undegraded":"2024-04-25T20:59:59.113156+0000","last_fullsized":"2024-04-25T20:59:59.113156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:12.375519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113274+0000","last_change":"2024-04-25T20:57:47.535216+0000","last_active":"2024-04-25T20:59:59.113274+0000","last_peered":"2024-04-25T20:59:59.113274+0000","last_clean":"2024-04-25T20:59:59.113274+0000","last_became_active":"2024-04-25T20:57:47.530348+0000","last_became_peered":"2024-04-25T20:57:47.530348+0000","last_unstale":"2024-04-25T20:59:59.113274+0000","last_undegraded":"2024-04-25T20:59:59.113274+0000","last_fullsized":"2024-04-25T20:59:59.113274+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:01.373541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042321+0000","last_change":"2024-04-25T20:56:51.277166+0000","last_active":"2024-04-25T20:59:59.042321+0000","last_peered":"2024-04-25T20:59:59.042321+0000","last_clean":"2024-04-25T20:59:59.042321+0000","last_became_active":"2024-04-25T20:56:51.276840+0000","last_became_peered":"2024-04-25T20:56:51.276840+0000","last_unstale":"2024-04-25T20:59:59.042321+0000","last_undegraded":"2024-04-25T20:59:59.042321+0000","last_fullsized":"2024-04-25T20:59:59.042321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:52.756660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016118+0000","last_change":"2024-04-25T20:56:48.263426+0000","last_active":"2024-04-25T20:59:59.016118+0000","last_peered":"2024-04-25T20:59:59.016118+0000","last_clean":"2024-04-25T20:59:59.016118+0000","last_became_active":"2024-04-25T20:56:48.262832+0000","last_became_peered":"2024-04-25T20:56:48.262832+0000","last_unstale":"2024-04-25T20:59:59.016118+0000","last_undegraded":"2024-04-25T20:59:59.016118+0000","last_fullsized":"2024-04-25T20:59:59.016118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:09.362161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549131+0000","last_change":"2024-04-25T20:56:51.274267+0000","last_active":"2024-04-25T20:59:59.549131+0000","last_peered":"2024-04-25T20:59:59.549131+0000","last_clean":"2024-04-25T20:59:59.549131+0000","last_became_active":"2024-04-25T20:56:51.273807+0000","last_became_peered":"2024-04-25T20:56:51.273807+0000","last_unstale":"2024-04-25T20:59:59.549131+0000","last_undegraded":"2024-04-25T20:59:59.549131+0000","last_fullsized":"2024-04-25T20:59:59.549131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266148+0000","last_change":"2024-04-25T20:56:49.031011+0000","last_active":"2024-04-25T20:59:59.266148+0000","last_peered":"2024-04-25T20:59:59.266148+0000","last_clean":"2024-04-25T20:59:59.266148+0000","last_became_active":"2024-04-25T20:56:49.030522+0000","last_became_peered":"2024-04-25T20:56:49.030522+0000","last_unstale":"2024-04-25T20:59:59.266148+0000","last_undegraded":"2024-04-25T20:59:59.266148+0000","last_fullsized":"2024-04-25T20:59:59.266148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:00.915454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549501+0000","last_change":"2024-04-25T20:56:51.276989+0000","last_active":"2024-04-25T20:59:59.549501+0000","last_peered":"2024-04-25T20:59:59.549501+0000","last_clean":"2024-04-25T20:59:59.549501+0000","last_became_active":"2024-04-25T20:56:51.275417+0000","last_became_peered":"2024-04-25T20:56:51.275417+0000","last_unstale":"2024-04-25T20:59:59.549501+0000","last_undegraded":"2024-04-25T20:59:59.549501+0000","last_fullsized":"2024-04-25T20:59:59.549501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:27.516062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'39","reported_seq":75,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.223899+0000","last_change":"2024-04-25T20:56:48.253713+0000","last_active":"2024-04-25T20:59:59.223899+0000","last_peered":"2024-04-25T20:59:59.223899+0000","last_clean":"2024-04-25T20:59:59.223899+0000","last_became_active":"2024-04-25T20:56:48.253215+0000","last_became_peered":"2024-04-25T20:56:48.253215+0000","last_unstale":"2024-04-25T20:59:59.223899+0000","last_undegraded":"2024-04-25T20:59:59.223899+0000","last_fullsized":"2024-04-25T20:59:59.223899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:57.946840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549273+0000","last_change":"2024-04-25T20:56:51.274691+0000","last_active":"2024-04-25T20:59:59.549273+0000","last_peered":"2024-04-25T20:59:59.549273+0000","last_clean":"2024-04-25T20:59:59.549273+0000","last_became_active":"2024-04-25T20:56:51.274350+0000","last_became_peered":"2024-04-25T20:56:51.274350+0000","last_unstale":"2024-04-25T20:59:59.549273+0000","last_undegraded":"2024-04-25T20:59:59.549273+0000","last_fullsized":"2024-04-25T20:59:59.549273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:52.375409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039075+0000","last_change":"2024-04-25T20:56:48.251616+0000","last_active":"2024-04-25T20:59:59.039075+0000","last_peered":"2024-04-25T20:59:59.039075+0000","last_clean":"2024-04-25T20:59:59.039075+0000","last_became_active":"2024-04-25T20:56:48.251248+0000","last_became_peered":"2024-04-25T20:56:48.251248+0000","last_unstale":"2024-04-25T20:59:59.039075+0000","last_undegraded":"2024-04-25T20:59:59.039075+0000","last_fullsized":"2024-04-25T20:59:59.039075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:51.032356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549388+0000","last_change":"2024-04-25T20:56:51.278372+0000","last_active":"2024-04-25T20:59:59.549388+0000","last_peered":"2024-04-25T20:59:59.549388+0000","last_clean":"2024-04-25T20:59:59.549388+0000","last_became_active":"2024-04-25T20:56:51.277989+0000","last_became_peered":"2024-04-25T20:56:51.277989+0000","last_unstale":"2024-04-25T20:59:59.549388+0000","last_undegraded":"2024-04-25T20:59:59.549388+0000","last_fullsized":"2024-04-25T20:59:59.549388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.017713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113230+0000","last_change":"2024-04-25T20:56:51.276270+0000","last_active":"2024-04-25T20:59:59.113230+0000","last_peered":"2024-04-25T20:59:59.113230+0000","last_clean":"2024-04-25T20:59:59.113230+0000","last_became_active":"2024-04-25T20:56:51.275948+0000","last_became_peered":"2024-04-25T20:56:51.275948+0000","last_unstale":"2024-04-25T20:59:59.113230+0000","last_undegraded":"2024-04-25T20:59:59.113230+0000","last_fullsized":"2024-04-25T20:59:59.113230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:33.548439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265845+0000","last_change":"2024-04-25T20:56:51.283931+0000","last_active":"2024-04-25T20:59:59.265845+0000","last_peered":"2024-04-25T20:59:59.265845+0000","last_clean":"2024-04-25T20:59:59.265845+0000","last_became_active":"2024-04-25T20:56:51.283120+0000","last_became_peered":"2024-04-25T20:56:51.283120+0000","last_unstale":"2024-04-25T20:59:59.265845+0000","last_undegraded":"2024-04-25T20:59:59.265845+0000","last_fullsized":"2024-04-25T20:59:59.265845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:26.676750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265827+0000","last_change":"2024-04-25T20:56:51.286013+0000","last_active":"2024-04-25T20:59:59.265827+0000","last_peered":"2024-04-25T20:59:59.265827+0000","last_clean":"2024-04-25T20:59:59.265827+0000","last_became_active":"2024-04-25T20:56:51.285645+0000","last_became_peered":"2024-04-25T20:56:51.285645+0000","last_unstale":"2024-04-25T20:59:59.265827+0000","last_undegraded":"2024-04-25T20:59:59.265827+0000","last_fullsized":"2024-04-25T20:59:59.265827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:51.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514609+0000","last_change":"2024-04-25T20:56:51.272567+0000","last_active":"2024-04-25T20:59:59.514609+0000","last_peered":"2024-04-25T20:59:59.514609+0000","last_clean":"2024-04-25T20:59:59.514609+0000","last_became_active":"2024-04-25T20:56:51.271427+0000","last_became_peered":"2024-04-25T20:56:51.271427+0000","last_unstale":"2024-04-25T20:59:59.514609+0000","last_undegraded":"2024-04-25T20:59:59.514609+0000","last_fullsized":"2024-04-25T20:59:59.514609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:19.945982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038896+0000","last_change":"2024-04-25T20:56:51.272448+0000","last_active":"2024-04-25T20:59:59.038896+0000","last_peered":"2024-04-25T20:59:59.038896+0000","last_clean":"2024-04-25T20:59:59.038896+0000","last_became_active":"2024-04-25T20:56:51.271965+0000","last_became_peered":"2024-04-25T20:56:51.271965+0000","last_unstale":"2024-04-25T20:59:59.038896+0000","last_undegraded":"2024-04-25T20:59:59.038896+0000","last_fullsized":"2024-04-25T20:59:59.038896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:03.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113416+0000","last_change":"2024-04-25T20:56:51.280260+0000","last_active":"2024-04-25T20:59:59.113416+0000","last_peered":"2024-04-25T20:59:59.113416+0000","last_clean":"2024-04-25T20:59:59.113416+0000","last_became_active":"2024-04-25T20:56:51.279864+0000","last_became_peered":"2024-04-25T20:56:51.279864+0000","last_unstale":"2024-04-25T20:59:59.113416+0000","last_undegraded":"2024-04-25T20:59:59.113416+0000","last_fullsized":"2024-04-25T20:59:59.113416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.136824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266115+0000","last_change":"2024-04-25T20:56:51.279041+0000","last_active":"2024-04-25T20:59:59.266115+0000","last_peered":"2024-04-25T20:59:59.266115+0000","last_clean":"2024-04-25T20:59:59.266115+0000","last_became_active":"2024-04-25T20:56:51.278302+0000","last_became_peered":"2024-04-25T20:56:51.278302+0000","last_unstale":"2024-04-25T20:59:59.266115+0000","last_undegraded":"2024-04-25T20:59:59.266115+0000","last_fullsized":"2024-04-25T20:59:59.266115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:02.510919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041907+0000","last_change":"2024-04-25T20:56:51.275618+0000","last_active":"2024-04-25T20:59:59.041907+0000","last_peered":"2024-04-25T20:59:59.041907+0000","last_clean":"2024-04-25T20:59:59.041907+0000","last_became_active":"2024-04-25T20:56:51.275249+0000","last_became_peered":"2024-04-25T20:56:51.275249+0000","last_unstale":"2024-04-25T20:59:59.041907+0000","last_undegraded":"2024-04-25T20:59:59.041907+0000","last_fullsized":"2024-04-25T20:59:59.041907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:01.793083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119251+0000","last_change":"2024-04-25T20:56:51.275868+0000","last_active":"2024-04-25T20:59:59.119251+0000","last_peered":"2024-04-25T20:59:59.119251+0000","last_clean":"2024-04-25T20:59:59.119251+0000","last_became_active":"2024-04-25T20:56:51.275544+0000","last_became_peered":"2024-04-25T20:56:51.275544+0000","last_unstale":"2024-04-25T20:59:59.119251+0000","last_undegraded":"2024-04-25T20:59:59.119251+0000","last_fullsized":"2024-04-25T20:59:59.119251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:31.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041380+0000","last_change":"2024-04-25T20:56:51.274963+0000","last_active":"2024-04-25T20:59:59.041380+0000","last_peered":"2024-04-25T20:59:59.041380+0000","last_clean":"2024-04-25T20:59:59.041380+0000","last_became_active":"2024-04-25T20:56:51.274597+0000","last_became_peered":"2024-04-25T20:56:51.274597+0000","last_unstale":"2024-04-25T20:59:59.041380+0000","last_undegraded":"2024-04-25T20:59:59.041380+0000","last_fullsized":"2024-04-25T20:59:59.041380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:14.464560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266007+0000","last_change":"2024-04-25T20:56:51.279602+0000","last_active":"2024-04-25T20:59:59.266007+0000","last_peered":"2024-04-25T20:59:59.266007+0000","last_clean":"2024-04-25T20:59:59.266007+0000","last_became_active":"2024-04-25T20:56:51.279221+0000","last_became_peered":"2024-04-25T20:56:51.279221+0000","last_unstale":"2024-04-25T20:59:59.266007+0000","last_undegraded":"2024-04-25T20:59:59.266007+0000","last_fullsized":"2024-04-25T20:59:59.266007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:26.960492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039002+0000","last_change":"2024-04-25T20:56:51.280465+0000","last_active":"2024-04-25T20:59:59.039002+0000","last_peered":"2024-04-25T20:59:59.039002+0000","last_clean":"2024-04-25T20:59:59.039002+0000","last_became_active":"2024-04-25T20:56:51.280168+0000","last_became_peered":"2024-04-25T20:56:51.280168+0000","last_unstale":"2024-04-25T20:59:59.039002+0000","last_undegraded":"2024-04-25T20:59:59.039002+0000","last_fullsized":"2024-04-25T20:59:59.039002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.211113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113237+0000","last_change":"2024-04-25T20:56:51.283084+0000","last_active":"2024-04-25T20:59:59.113237+0000","last_peered":"2024-04-25T20:59:59.113237+0000","last_clean":"2024-04-25T20:59:59.113237+0000","last_became_active":"2024-04-25T20:56:51.282692+0000","last_became_peered":"2024-04-25T20:56:51.282692+0000","last_unstale":"2024-04-25T20:59:59.113237+0000","last_undegraded":"2024-04-25T20:59:59.113237+0000","last_fullsized":"2024-04-25T20:59:59.113237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:13:27.527112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596857,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1159,"num_read_kb":690,"num_write":293,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1790571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351029,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1409024,"data_stored":886419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579963,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37328,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707824,"statfs":{"total":95995035648,"available":95956811776,"internally_reserved":0,"allocated":1855488,"data_stored":1283401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776191,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1425408,"data_stored":886655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":493359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299464,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1626112,"data_stored":1086270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462990,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1019904,"data_stored":492828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":559680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":494099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":561562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1017955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920855,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":485141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724627,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33088,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712064,"statfs":{"total":95995035648,"available":95961153536,"internally_reserved":0,"allocated":1773568,"data_stored":1216714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:00:05.102 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:00:05.355 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:05.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:05.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:05.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:05.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:05.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:05.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:05.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:05.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: Updating smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:00:06.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:06 smithi047 ceph-mon[29195]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-25T21:00:06.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:06 smithi186 ceph-mon[32529]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-25T21:00:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 -- 172.21.15.47:0/2336712642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74108a70 msgr2=0x7fbf74108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 --2- 172.21.15.47:0/2336712642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74108a70 0x7fbf74108e50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009990 tx=0x7fbf70009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 -- 172.21.15.47:0/2336712642 shutdown_connections 2024-04-25T21:00:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 --2- 172.21.15.47:0/2336712642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74108a70 0x7fbf74108e50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 --2- 172.21.15.47:0/2336712642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74103480 0x7fbf7410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 --2- 172.21.15.47:0/2336712642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf74102ac0 0x7fbf74102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 -- 172.21.15.47:0/2336712642 >> 172.21.15.47:0/2336712642 conn(0x7fbf74079bb0 msgr2=0x7fbf74079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 -- 172.21.15.47:0/2336712642 shutdown_connections 2024-04-25T21:00:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 -- 172.21.15.47:0/2336712642 wait complete. 2024-04-25T21:00:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 Processor -- start 2024-04-25T21:00:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.017+0000 7fbf7c223700 1 -- start start 2024-04-25T21:00:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf7c223700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74102ac0 0x7fbf74074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf7c223700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74103480 0x7fbf74075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf7c223700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf74108a70 0x7fbf740757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf7c223700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112c80 con 0x7fbf74102ac0 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf7c223700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112b00 con 0x7fbf74103480 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf7c223700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112e00 con 0x7fbf74108a70 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74102ac0 0x7fbf74074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74102ac0 0x7fbf74074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42266/0 (socket says 172.21.15.47:42266) 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 -- 172.21.15.47:0/958593919 learned_addr learned my addr 172.21.15.47:0/958593919 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf797be700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74103480 0x7fbf74075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf7a7c0700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf74108a70 0x7fbf740757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 -- 172.21.15.47:0/958593919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf74108a70 msgr2=0x7fbf740757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf74108a70 0x7fbf740757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 -- 172.21.15.47:0/958593919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74103480 msgr2=0x7fbf74075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74103480 0x7fbf74075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 -- 172.21.15.47:0/958593919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf741a7610 con 0x7fbf74102ac0 2024-04-25T21:00:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.018+0000 7fbf79fbf700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74102ac0 0x7fbf74074d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf70000c00 tx=0x7fbf700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.019+0000 7fbf6affd700 1 -- 172.21.15.47:0/958593919 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70027070 con 0x7fbf74102ac0 2024-04-25T21:00:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.019+0000 7fbf6affd700 1 -- 172.21.15.47:0/958593919 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf700040e0 con 0x7fbf74102ac0 2024-04-25T21:00:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.019+0000 7fbf6affd700 1 -- 172.21.15.47:0/958593919 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000f460 con 0x7fbf74102ac0 2024-04-25T21:00:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.019+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf741a78a0 con 0x7fbf74102ac0 2024-04-25T21:00:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.019+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741a7d80 con 0x7fbf74102ac0 2024-04-25T21:00:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.020+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf74069730 con 0x7fbf74102ac0 2024-04-25T21:00:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.021+0000 7fbf6affd700 1 -- 172.21.15.47:0/958593919 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7fbf7002a070 con 0x7fbf74102ac0 2024-04-25T21:00:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.021+0000 7fbf6affd700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbf600a3ed0 0x7fbf600a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.022+0000 7fbf797be700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbf600a3ed0 0x7fbf600a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.022+0000 7fbf6affd700 1 -- 172.21.15.47:0/958593919 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fbf700d19a0 con 0x7fbf74102ac0 2024-04-25T21:00:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.022+0000 7fbf797be700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbf600a3ed0 0x7fbf600a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf7407a200 tx=0x7fbf64009190 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:07.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.026+0000 7fbf6affd700 1 -- 172.21.15.47:0/958593919 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf7005eff0 con 0x7fbf74102ac0 2024-04-25T21:00:07.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:06 smithi092 ceph-mon[32524]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-25T21:00:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.235+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf74062e80 con 0x7fbf600a3ed0 2024-04-25T21:00:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.242+0000 7fbf6affd700 1 -- 172.21.15.47:0/958593919 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257767 (secure 0 0 0) 0x7fbf74062e80 con 0x7fbf600a3ed0 2024-04-25T21:00:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbf600a3ed0 msgr2=0x7fbf600a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbf600a3ed0 0x7fbf600a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf7407a200 tx=0x7fbf64009190 comp rx=0 tx=0).stop 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74102ac0 msgr2=0x7fbf74074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74102ac0 0x7fbf74074d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf70000c00 tx=0x7fbf700047b0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 shutdown_connections 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbf600a3ed0 0x7fbf600a6390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74102ac0 0x7fbf74074d40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf74108a70 0x7fbf740757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 --2- 172.21.15.47:0/958593919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74103480 0x7fbf74075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 >> 172.21.15.47:0/958593919 conn(0x7fbf74079bb0 msgr2=0x7fbf74101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 shutdown_connections 2024-04-25T21:00:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:07.245+0000 7fbf7c223700 1 -- 172.21.15.47:0/958593919 wait complete. 2024-04-25T21:00:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-25T21:00:07.903 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-04-25T21:00:07.029617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387002,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1681,"num_read_kb":1269,"num_write":626,"num_write_kb":5297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":961,"ondisk_log_size":961,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365716,"kb_used_data":16176,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124576108,"statfs":{"total":1151940427776,"available":1151565934592,"internally_reserved":0,"allocated":16564224,"data_stored":10072458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199543,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":386,"num_read_kb":460,"num_write":150,"num_write_kb":919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.993619"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113432+0000","last_change":"2024-04-25T20:56:51.283722+0000","last_active":"2024-04-25T20:59:59.113432+0000","last_peered":"2024-04-25T20:59:59.113432+0000","last_clean":"2024-04-25T20:59:59.113432+0000","last_became_active":"2024-04-25T20:56:51.283359+0000","last_became_peered":"2024-04-25T20:56:51.283359+0000","last_unstale":"2024-04-25T20:59:59.113432+0000","last_undegraded":"2024-04-25T20:59:59.113432+0000","last_fullsized":"2024-04-25T20:59:59.113432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:47.314687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514767+0000","last_change":"2024-04-25T20:56:51.273199+0000","last_active":"2024-04-25T20:59:59.514767+0000","last_peered":"2024-04-25T20:59:59.514767+0000","last_clean":"2024-04-25T20:59:59.514767+0000","last_became_active":"2024-04-25T20:56:51.272838+0000","last_became_peered":"2024-04-25T20:56:51.272838+0000","last_unstale":"2024-04-25T20:59:59.514767+0000","last_undegraded":"2024-04-25T20:59:59.514767+0000","last_fullsized":"2024-04-25T20:59:59.514767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:58.033315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514639+0000","last_change":"2024-04-25T20:56:51.281461+0000","last_active":"2024-04-25T20:59:59.514639+0000","last_peered":"2024-04-25T20:59:59.514639+0000","last_clean":"2024-04-25T20:59:59.514639+0000","last_became_active":"2024-04-25T20:56:51.281141+0000","last_became_peered":"2024-04-25T20:56:51.281141+0000","last_unstale":"2024-04-25T20:59:59.514639+0000","last_undegraded":"2024-04-25T20:59:59.514639+0000","last_fullsized":"2024-04-25T20:59:59.514639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041716+0000","last_change":"2024-04-25T20:56:51.283187+0000","last_active":"2024-04-25T20:59:59.041716+0000","last_peered":"2024-04-25T20:59:59.041716+0000","last_clean":"2024-04-25T20:59:59.041716+0000","last_became_active":"2024-04-25T20:56:51.282640+0000","last_became_peered":"2024-04-25T20:56:51.282640+0000","last_unstale":"2024-04-25T20:59:59.041716+0000","last_undegraded":"2024-04-25T20:59:59.041716+0000","last_fullsized":"2024-04-25T20:59:59.041716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:15.588694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016083+0000","last_change":"2024-04-25T20:56:51.273204+0000","last_active":"2024-04-25T20:59:59.016083+0000","last_peered":"2024-04-25T20:59:59.016083+0000","last_clean":"2024-04-25T20:59:59.016083+0000","last_became_active":"2024-04-25T20:56:51.272126+0000","last_became_peered":"2024-04-25T20:56:51.272126+0000","last_unstale":"2024-04-25T20:59:59.016083+0000","last_undegraded":"2024-04-25T20:59:59.016083+0000","last_fullsized":"2024-04-25T20:59:59.016083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:31.684294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016332+0000","last_change":"2024-04-25T20:56:51.270762+0000","last_active":"2024-04-25T20:59:59.016332+0000","last_peered":"2024-04-25T20:59:59.016332+0000","last_clean":"2024-04-25T20:59:59.016332+0000","last_became_active":"2024-04-25T20:56:51.270444+0000","last_became_peered":"2024-04-25T20:56:51.270444+0000","last_unstale":"2024-04-25T20:59:59.016332+0000","last_undegraded":"2024-04-25T20:59:59.016332+0000","last_fullsized":"2024-04-25T20:59:59.016332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:00.155417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549002+0000","last_change":"2024-04-25T20:56:51.276029+0000","last_active":"2024-04-25T20:59:59.549002+0000","last_peered":"2024-04-25T20:59:59.549002+0000","last_clean":"2024-04-25T20:59:59.549002+0000","last_became_active":"2024-04-25T20:56:51.275272+0000","last_became_peered":"2024-04-25T20:56:51.275272+0000","last_unstale":"2024-04-25T20:59:59.549002+0000","last_undegraded":"2024-04-25T20:59:59.549002+0000","last_fullsized":"2024-04-25T20:59:59.549002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.428444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041346+0000","last_change":"2024-04-25T20:56:51.273769+0000","last_active":"2024-04-25T20:59:59.041346+0000","last_peered":"2024-04-25T20:59:59.041346+0000","last_clean":"2024-04-25T20:59:59.041346+0000","last_became_active":"2024-04-25T20:56:51.273412+0000","last_became_peered":"2024-04-25T20:56:51.273412+0000","last_unstale":"2024-04-25T20:59:59.041346+0000","last_undegraded":"2024-04-25T20:59:59.041346+0000","last_fullsized":"2024-04-25T20:59:59.041346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.324332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266500+0000","last_change":"2024-04-25T20:56:51.277097+0000","last_active":"2024-04-25T20:59:59.266500+0000","last_peered":"2024-04-25T20:59:59.266500+0000","last_clean":"2024-04-25T20:59:59.266500+0000","last_became_active":"2024-04-25T20:56:51.276768+0000","last_became_peered":"2024-04-25T20:56:51.276768+0000","last_unstale":"2024-04-25T20:59:59.266500+0000","last_undegraded":"2024-04-25T20:59:59.266500+0000","last_fullsized":"2024-04-25T20:59:59.266500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:23.276470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016837+0000","last_change":"2024-04-25T20:56:51.279046+0000","last_active":"2024-04-25T20:59:59.016837+0000","last_peered":"2024-04-25T20:59:59.016837+0000","last_clean":"2024-04-25T20:59:59.016837+0000","last_became_active":"2024-04-25T20:56:51.278701+0000","last_became_peered":"2024-04-25T20:56:51.278701+0000","last_unstale":"2024-04-25T20:59:59.016837+0000","last_undegraded":"2024-04-25T20:59:59.016837+0000","last_fullsized":"2024-04-25T20:59:59.016837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.370482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549375+0000","last_change":"2024-04-25T20:56:51.271593+0000","last_active":"2024-04-25T20:59:59.549375+0000","last_peered":"2024-04-25T20:59:59.549375+0000","last_clean":"2024-04-25T20:59:59.549375+0000","last_became_active":"2024-04-25T20:56:51.271224+0000","last_became_peered":"2024-04-25T20:56:51.271224+0000","last_unstale":"2024-04-25T20:59:59.549375+0000","last_undegraded":"2024-04-25T20:59:59.549375+0000","last_fullsized":"2024-04-25T20:59:59.549375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:46.847929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041553+0000","last_change":"2024-04-25T20:56:51.278281+0000","last_active":"2024-04-25T20:59:59.041553+0000","last_peered":"2024-04-25T20:59:59.041553+0000","last_clean":"2024-04-25T20:59:59.041553+0000","last_became_active":"2024-04-25T20:56:51.277947+0000","last_became_peered":"2024-04-25T20:56:51.277947+0000","last_unstale":"2024-04-25T20:59:59.041553+0000","last_undegraded":"2024-04-25T20:59:59.041553+0000","last_fullsized":"2024-04-25T20:59:59.041553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:02.509296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039058+0000","last_change":"2024-04-25T20:56:51.273784+0000","last_active":"2024-04-25T20:59:59.039058+0000","last_peered":"2024-04-25T20:59:59.039058+0000","last_clean":"2024-04-25T20:59:59.039058+0000","last_became_active":"2024-04-25T20:56:51.273489+0000","last_became_peered":"2024-04-25T20:56:51.273489+0000","last_unstale":"2024-04-25T20:59:59.039058+0000","last_undegraded":"2024-04-25T20:59:59.039058+0000","last_fullsized":"2024-04-25T20:59:59.039058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:00.765186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514418+0000","last_change":"2024-04-25T20:56:51.269829+0000","last_active":"2024-04-25T20:59:59.514418+0000","last_peered":"2024-04-25T20:59:59.514418+0000","last_clean":"2024-04-25T20:59:59.514418+0000","last_became_active":"2024-04-25T20:56:51.269446+0000","last_became_peered":"2024-04-25T20:56:51.269446+0000","last_unstale":"2024-04-25T20:59:59.514418+0000","last_undegraded":"2024-04-25T20:59:59.514418+0000","last_fullsized":"2024-04-25T20:59:59.514418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:12.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514335+0000","last_change":"2024-04-25T20:56:51.278117+0000","last_active":"2024-04-25T20:59:59.514335+0000","last_peered":"2024-04-25T20:59:59.514335+0000","last_clean":"2024-04-25T20:59:59.514335+0000","last_became_active":"2024-04-25T20:56:51.277596+0000","last_became_peered":"2024-04-25T20:56:51.277596+0000","last_unstale":"2024-04-25T20:59:59.514335+0000","last_undegraded":"2024-04-25T20:59:59.514335+0000","last_fullsized":"2024-04-25T20:59:59.514335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:28.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265776+0000","last_change":"2024-04-25T20:56:51.281575+0000","last_active":"2024-04-25T20:59:59.265776+0000","last_peered":"2024-04-25T20:59:59.265776+0000","last_clean":"2024-04-25T20:59:59.265776+0000","last_became_active":"2024-04-25T20:56:51.281217+0000","last_became_peered":"2024-04-25T20:56:51.281217+0000","last_unstale":"2024-04-25T20:59:59.265776+0000","last_undegraded":"2024-04-25T20:59:59.265776+0000","last_fullsized":"2024-04-25T20:59:59.265776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:05.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113547+0000","last_change":"2024-04-25T20:56:51.284608+0000","last_active":"2024-04-25T20:59:59.113547+0000","last_peered":"2024-04-25T20:59:59.113547+0000","last_clean":"2024-04-25T20:59:59.113547+0000","last_became_active":"2024-04-25T20:56:51.284257+0000","last_became_peered":"2024-04-25T20:56:51.284257+0000","last_unstale":"2024-04-25T20:59:59.113547+0000","last_undegraded":"2024-04-25T20:59:59.113547+0000","last_fullsized":"2024-04-25T20:59:59.113547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:44:59.184192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013207+0000","last_change":"2024-04-25T20:56:51.276770+0000","last_active":"2024-04-25T20:59:59.013207+0000","last_peered":"2024-04-25T20:59:59.013207+0000","last_clean":"2024-04-25T20:59:59.013207+0000","last_became_active":"2024-04-25T20:56:51.276431+0000","last_became_peered":"2024-04-25T20:56:51.276431+0000","last_unstale":"2024-04-25T20:59:59.013207+0000","last_undegraded":"2024-04-25T20:59:59.013207+0000","last_fullsized":"2024-04-25T20:59:59.013207+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:14:25.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265999+0000","last_change":"2024-04-25T20:56:51.283728+0000","last_active":"2024-04-25T20:59:59.265999+0000","last_peered":"2024-04-25T20:59:59.265999+0000","last_clean":"2024-04-25T20:59:59.265999+0000","last_became_active":"2024-04-25T20:56:51.283370+0000","last_became_peered":"2024-04-25T20:56:51.283370+0000","last_unstale":"2024-04-25T20:59:59.265999+0000","last_undegraded":"2024-04-25T20:59:59.265999+0000","last_fullsized":"2024-04-25T20:59:59.265999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:18.042769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012950+0000","last_change":"2024-04-25T20:56:48.259737+0000","last_active":"2024-04-25T20:59:59.012950+0000","last_peered":"2024-04-25T20:59:59.012950+0000","last_clean":"2024-04-25T20:59:59.012950+0000","last_became_active":"2024-04-25T20:56:48.259442+0000","last_became_peered":"2024-04-25T20:56:48.259442+0000","last_unstale":"2024-04-25T20:59:59.012950+0000","last_undegraded":"2024-04-25T20:59:59.012950+0000","last_fullsized":"2024-04-25T20:59:59.012950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:23.191536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042189+0000","last_change":"2024-04-25T20:56:51.274510+0000","last_active":"2024-04-25T20:59:59.042189+0000","last_peered":"2024-04-25T20:59:59.042189+0000","last_clean":"2024-04-25T20:59:59.042189+0000","last_became_active":"2024-04-25T20:56:51.274137+0000","last_became_peered":"2024-04-25T20:56:51.274137+0000","last_unstale":"2024-04-25T20:59:59.042189+0000","last_undegraded":"2024-04-25T20:59:59.042189+0000","last_fullsized":"2024-04-25T20:59:59.042189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:05.903617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042405+0000","last_change":"2024-04-25T20:56:48.947850+0000","last_active":"2024-04-25T20:59:59.042405+0000","last_peered":"2024-04-25T20:59:59.042405+0000","last_clean":"2024-04-25T20:59:59.042405+0000","last_became_active":"2024-04-25T20:56:48.946160+0000","last_became_peered":"2024-04-25T20:56:48.946160+0000","last_unstale":"2024-04-25T20:59:59.042405+0000","last_undegraded":"2024-04-25T20:59:59.042405+0000","last_fullsized":"2024-04-25T20:59:59.042405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:32.692390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.515020+0000","last_change":"2024-04-25T20:56:51.280478+0000","last_active":"2024-04-25T20:59:59.515020+0000","last_peered":"2024-04-25T20:59:59.515020+0000","last_clean":"2024-04-25T20:59:59.515020+0000","last_became_active":"2024-04-25T20:56:51.280173+0000","last_became_peered":"2024-04-25T20:56:51.280173+0000","last_unstale":"2024-04-25T20:59:59.515020+0000","last_undegraded":"2024-04-25T20:59:59.515020+0000","last_fullsized":"2024-04-25T20:59:59.515020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:48.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016506+0000","last_change":"2024-04-25T20:57:47.535264+0000","last_active":"2024-04-25T20:59:59.016506+0000","last_peered":"2024-04-25T20:59:59.016506+0000","last_clean":"2024-04-25T20:59:59.016506+0000","last_became_active":"2024-04-25T20:57:47.531152+0000","last_became_peered":"2024-04-25T20:57:47.531152+0000","last_unstale":"2024-04-25T20:59:59.016506+0000","last_undegraded":"2024-04-25T20:59:59.016506+0000","last_fullsized":"2024-04-25T20:59:59.016506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:19.085455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265973+0000","last_change":"2024-04-25T20:56:51.278383+0000","last_active":"2024-04-25T20:59:59.265973+0000","last_peered":"2024-04-25T20:59:59.265973+0000","last_clean":"2024-04-25T20:59:59.265973+0000","last_became_active":"2024-04-25T20:56:51.277603+0000","last_became_peered":"2024-04-25T20:56:51.277603+0000","last_unstale":"2024-04-25T20:59:59.265973+0000","last_undegraded":"2024-04-25T20:59:59.265973+0000","last_fullsized":"2024-04-25T20:59:59.265973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:12.197940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016420+0000","last_change":"2024-04-25T20:56:48.258869+0000","last_active":"2024-04-25T20:59:59.016420+0000","last_peered":"2024-04-25T20:59:59.016420+0000","last_clean":"2024-04-25T20:59:59.016420+0000","last_became_active":"2024-04-25T20:56:48.258414+0000","last_became_peered":"2024-04-25T20:56:48.258414+0000","last_unstale":"2024-04-25T20:59:59.016420+0000","last_undegraded":"2024-04-25T20:59:59.016420+0000","last_fullsized":"2024-04-25T20:59:59.016420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:09.889936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013004+0000","last_change":"2024-04-25T20:57:48.300637+0000","last_active":"2024-04-25T20:59:59.013004+0000","last_peered":"2024-04-25T20:59:59.013004+0000","last_clean":"2024-04-25T20:59:59.013004+0000","last_became_active":"2024-04-25T20:57:48.300285+0000","last_became_peered":"2024-04-25T20:57:48.300285+0000","last_unstale":"2024-04-25T20:59:59.013004+0000","last_undegraded":"2024-04-25T20:59:59.013004+0000","last_fullsized":"2024-04-25T20:59:59.013004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:31.810671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:04.017318+0000","last_change":"2024-04-25T20:56:48.262543+0000","last_active":"2024-04-25T21:00:04.017318+0000","last_peered":"2024-04-25T21:00:04.017318+0000","last_clean":"2024-04-25T21:00:04.017318+0000","last_became_active":"2024-04-25T20:56:48.262086+0000","last_became_peered":"2024-04-25T20:56:48.262086+0000","last_unstale":"2024-04-25T21:00:04.017318+0000","last_undegraded":"2024-04-25T21:00:04.017318+0000","last_fullsized":"2024-04-25T21:00:04.017318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:43.722131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015962+0000","last_change":"2024-04-25T20:56:51.273500+0000","last_active":"2024-04-25T20:59:59.015962+0000","last_peered":"2024-04-25T20:59:59.015962+0000","last_clean":"2024-04-25T20:59:59.015962+0000","last_became_active":"2024-04-25T20:56:51.272882+0000","last_became_peered":"2024-04-25T20:56:51.272882+0000","last_unstale":"2024-04-25T20:59:59.015962+0000","last_undegraded":"2024-04-25T20:59:59.015962+0000","last_fullsized":"2024-04-25T20:59:59.015962+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:05.682258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016218+0000","last_change":"2024-04-25T20:56:48.255225+0000","last_active":"2024-04-25T20:59:59.016218+0000","last_peered":"2024-04-25T20:59:59.016218+0000","last_clean":"2024-04-25T20:59:59.016218+0000","last_became_active":"2024-04-25T20:56:48.254644+0000","last_became_peered":"2024-04-25T20:56:48.254644+0000","last_unstale":"2024-04-25T20:59:59.016218+0000","last_undegraded":"2024-04-25T20:59:59.016218+0000","last_fullsized":"2024-04-25T20:59:59.016218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:35.724612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119382+0000","last_change":"2024-04-25T20:56:51.270709+0000","last_active":"2024-04-25T20:59:59.119382+0000","last_peered":"2024-04-25T20:59:59.119382+0000","last_clean":"2024-04-25T20:59:59.119382+0000","last_became_active":"2024-04-25T20:56:51.270295+0000","last_became_peered":"2024-04-25T20:56:51.270295+0000","last_unstale":"2024-04-25T20:59:59.119382+0000","last_undegraded":"2024-04-25T20:59:59.119382+0000","last_fullsized":"2024-04-25T20:59:59.119382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:41.998953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038784+0000","last_change":"2024-04-25T20:56:48.256498+0000","last_active":"2024-04-25T20:59:59.038784+0000","last_peered":"2024-04-25T20:59:59.038784+0000","last_clean":"2024-04-25T20:59:59.038784+0000","last_became_active":"2024-04-25T20:56:48.256140+0000","last_became_peered":"2024-04-25T20:56:48.256140+0000","last_unstale":"2024-04-25T20:59:59.038784+0000","last_undegraded":"2024-04-25T20:59:59.038784+0000","last_fullsized":"2024-04-25T20:59:59.038784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:41.458235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041426+0000","last_change":"2024-04-25T20:56:51.267575+0000","last_active":"2024-04-25T20:59:59.041426+0000","last_peered":"2024-04-25T20:59:59.041426+0000","last_clean":"2024-04-25T20:59:59.041426+0000","last_became_active":"2024-04-25T20:56:51.267265+0000","last_became_peered":"2024-04-25T20:56:51.267265+0000","last_unstale":"2024-04-25T20:59:59.041426+0000","last_undegraded":"2024-04-25T20:59:59.041426+0000","last_fullsized":"2024-04-25T20:59:59.041426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:30.760555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119218+0000","last_change":"2024-04-25T20:56:48.260747+0000","last_active":"2024-04-25T20:59:59.119218+0000","last_peered":"2024-04-25T20:59:59.119218+0000","last_clean":"2024-04-25T20:59:59.119218+0000","last_became_active":"2024-04-25T20:56:48.260339+0000","last_became_peered":"2024-04-25T20:56:48.260339+0000","last_unstale":"2024-04-25T20:59:59.119218+0000","last_undegraded":"2024-04-25T20:59:59.119218+0000","last_fullsized":"2024-04-25T20:59:59.119218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:19.769631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012748+0000","last_change":"2024-04-25T20:57:47.795125+0000","last_active":"2024-04-25T20:59:59.012748+0000","last_peered":"2024-04-25T20:59:59.012748+0000","last_clean":"2024-04-25T20:59:59.012748+0000","last_became_active":"2024-04-25T20:57:47.794642+0000","last_became_peered":"2024-04-25T20:57:47.794642+0000","last_unstale":"2024-04-25T20:59:59.012748+0000","last_undegraded":"2024-04-25T20:59:59.012748+0000","last_fullsized":"2024-04-25T20:59:59.012748+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:15.443731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":210,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:04.016806+0000","last_change":"2024-04-25T20:56:48.256113+0000","last_active":"2024-04-25T21:00:04.016806+0000","last_peered":"2024-04-25T21:00:04.016806+0000","last_clean":"2024-04-25T21:00:04.016806+0000","last_became_active":"2024-04-25T20:56:48.255644+0000","last_became_peered":"2024-04-25T20:56:48.255644+0000","last_unstale":"2024-04-25T21:00:04.016806+0000","last_undegraded":"2024-04-25T21:00:04.016806+0000","last_fullsized":"2024-04-25T21:00:04.016806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:12.100099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":177,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266407+0000","last_change":"2024-04-25T20:56:51.276990+0000","last_active":"2024-04-25T20:59:59.266407+0000","last_peered":"2024-04-25T20:59:59.266407+0000","last_clean":"2024-04-25T20:59:59.266407+0000","last_became_active":"2024-04-25T20:56:51.276638+0000","last_became_peered":"2024-04-25T20:56:51.276638+0000","last_unstale":"2024-04-25T20:59:59.266407+0000","last_undegraded":"2024-04-25T20:59:59.266407+0000","last_fullsized":"2024-04-25T20:59:59.266407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:06.645278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549502+0000","last_change":"2024-04-25T20:56:48.263216+0000","last_active":"2024-04-25T20:59:59.549502+0000","last_peered":"2024-04-25T20:59:59.549502+0000","last_clean":"2024-04-25T20:59:59.549502+0000","last_became_active":"2024-04-25T20:56:48.262854+0000","last_became_peered":"2024-04-25T20:56:48.262854+0000","last_unstale":"2024-04-25T20:59:59.549502+0000","last_undegraded":"2024-04-25T20:59:59.549502+0000","last_fullsized":"2024-04-25T20:59:59.549502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:00.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514957+0000","last_change":"2024-04-25T20:56:51.278479+0000","last_active":"2024-04-25T20:59:59.514957+0000","last_peered":"2024-04-25T20:59:59.514957+0000","last_clean":"2024-04-25T20:59:59.514957+0000","last_became_active":"2024-04-25T20:56:51.276285+0000","last_became_peered":"2024-04-25T20:56:51.276285+0000","last_unstale":"2024-04-25T20:59:59.514957+0000","last_undegraded":"2024-04-25T20:59:59.514957+0000","last_fullsized":"2024-04-25T20:59:59.514957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:14.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119578+0000","last_change":"2024-04-25T20:56:48.946808+0000","last_active":"2024-04-25T20:59:59.119578+0000","last_peered":"2024-04-25T20:59:59.119578+0000","last_clean":"2024-04-25T20:59:59.119578+0000","last_became_active":"2024-04-25T20:56:48.946421+0000","last_became_peered":"2024-04-25T20:56:48.946421+0000","last_unstale":"2024-04-25T20:59:59.119578+0000","last_undegraded":"2024-04-25T20:59:59.119578+0000","last_fullsized":"2024-04-25T20:59:59.119578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:47.763950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266116+0000","last_change":"2024-04-25T20:56:51.274761+0000","last_active":"2024-04-25T20:59:59.266116+0000","last_peered":"2024-04-25T20:59:59.266116+0000","last_clean":"2024-04-25T20:59:59.266116+0000","last_became_active":"2024-04-25T20:56:51.274452+0000","last_became_peered":"2024-04-25T20:56:51.274452+0000","last_unstale":"2024-04-25T20:59:59.266116+0000","last_undegraded":"2024-04-25T20:59:59.266116+0000","last_fullsized":"2024-04-25T20:59:59.266116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:32.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":437,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:05.018237+0000","last_change":"2024-04-25T20:56:48.258982+0000","last_active":"2024-04-25T21:00:05.018237+0000","last_peered":"2024-04-25T21:00:05.018237+0000","last_clean":"2024-04-25T21:00:05.018237+0000","last_became_active":"2024-04-25T20:56:48.258593+0000","last_became_peered":"2024-04-25T20:56:48.258593+0000","last_unstale":"2024-04-25T21:00:05.018237+0000","last_undegraded":"2024-04-25T21:00:05.018237+0000","last_fullsized":"2024-04-25T21:00:05.018237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:20.518202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69970,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":370,"num_read_kb":254,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016324+0000","last_change":"2024-04-25T20:56:51.276361+0000","last_active":"2024-04-25T20:59:59.016324+0000","last_peered":"2024-04-25T20:59:59.016324+0000","last_clean":"2024-04-25T20:59:59.016324+0000","last_became_active":"2024-04-25T20:56:51.276045+0000","last_became_peered":"2024-04-25T20:56:51.276045+0000","last_unstale":"2024-04-25T20:59:59.016324+0000","last_undegraded":"2024-04-25T20:59:59.016324+0000","last_fullsized":"2024-04-25T20:59:59.016324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:04.325068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266011+0000","last_change":"2024-04-25T20:56:48.252358+0000","last_active":"2024-04-25T20:59:59.266011+0000","last_peered":"2024-04-25T20:59:59.266011+0000","last_clean":"2024-04-25T20:59:59.266011+0000","last_became_active":"2024-04-25T20:56:48.252027+0000","last_became_peered":"2024-04-25T20:56:48.252027+0000","last_unstale":"2024-04-25T20:59:59.266011+0000","last_undegraded":"2024-04-25T20:59:59.266011+0000","last_fullsized":"2024-04-25T20:59:59.266011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:03.024487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038654+0000","last_change":"2024-04-25T20:56:51.279599+0000","last_active":"2024-04-25T20:59:59.038654+0000","last_peered":"2024-04-25T20:59:59.038654+0000","last_clean":"2024-04-25T20:59:59.038654+0000","last_became_active":"2024-04-25T20:56:51.279294+0000","last_became_peered":"2024-04-25T20:56:51.279294+0000","last_unstale":"2024-04-25T20:59:59.038654+0000","last_undegraded":"2024-04-25T20:59:59.038654+0000","last_fullsized":"2024-04-25T20:59:59.038654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119216+0000","last_change":"2024-04-25T20:56:48.261005+0000","last_active":"2024-04-25T20:59:59.119216+0000","last_peered":"2024-04-25T20:59:59.119216+0000","last_clean":"2024-04-25T20:59:59.119216+0000","last_became_active":"2024-04-25T20:56:48.260543+0000","last_became_peered":"2024-04-25T20:56:48.260543+0000","last_unstale":"2024-04-25T20:59:59.119216+0000","last_undegraded":"2024-04-25T20:59:59.119216+0000","last_fullsized":"2024-04-25T20:59:59.119216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:41:54.326036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549230+0000","last_change":"2024-04-25T20:56:51.271753+0000","last_active":"2024-04-25T20:59:59.549230+0000","last_peered":"2024-04-25T20:59:59.549230+0000","last_clean":"2024-04-25T20:59:59.549230+0000","last_became_active":"2024-04-25T20:56:51.271392+0000","last_became_peered":"2024-04-25T20:56:51.271392+0000","last_unstale":"2024-04-25T20:59:59.549230+0000","last_undegraded":"2024-04-25T20:59:59.549230+0000","last_fullsized":"2024-04-25T20:59:59.549230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:26.605647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":215,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:03.047987+0000","last_change":"2024-04-25T20:56:48.260729+0000","last_active":"2024-04-25T21:00:03.047987+0000","last_peered":"2024-04-25T21:00:03.047987+0000","last_clean":"2024-04-25T21:00:03.047987+0000","last_became_active":"2024-04-25T20:56:48.260377+0000","last_became_peered":"2024-04-25T20:56:48.260377+0000","last_unstale":"2024-04-25T21:00:03.047987+0000","last_undegraded":"2024-04-25T21:00:03.047987+0000","last_fullsized":"2024-04-25T21:00:03.047987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:21.582266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.548948+0000","last_change":"2024-04-25T20:56:51.275691+0000","last_active":"2024-04-25T20:59:59.548948+0000","last_peered":"2024-04-25T20:59:59.548948+0000","last_clean":"2024-04-25T20:59:59.548948+0000","last_became_active":"2024-04-25T20:56:51.275220+0000","last_became_peered":"2024-04-25T20:56:51.275220+0000","last_unstale":"2024-04-25T20:59:59.548948+0000","last_undegraded":"2024-04-25T20:59:59.548948+0000","last_fullsized":"2024-04-25T20:59:59.548948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:24.367413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015925+0000","last_change":"2024-04-25T20:56:48.946465+0000","last_active":"2024-04-25T20:59:59.015925+0000","last_peered":"2024-04-25T20:59:59.015925+0000","last_clean":"2024-04-25T20:59:59.015925+0000","last_became_active":"2024-04-25T20:56:48.946093+0000","last_became_peered":"2024-04-25T20:56:48.946093+0000","last_unstale":"2024-04-25T20:59:59.015925+0000","last_undegraded":"2024-04-25T20:59:59.015925+0000","last_fullsized":"2024-04-25T20:59:59.015925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:29.502073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119264+0000","last_change":"2024-04-25T20:56:51.278464+0000","last_active":"2024-04-25T20:59:59.119264+0000","last_peered":"2024-04-25T20:59:59.119264+0000","last_clean":"2024-04-25T20:59:59.119264+0000","last_became_active":"2024-04-25T20:56:51.278119+0000","last_became_peered":"2024-04-25T20:56:51.278119+0000","last_unstale":"2024-04-25T20:59:59.119264+0000","last_undegraded":"2024-04-25T20:59:59.119264+0000","last_fullsized":"2024-04-25T20:59:59.119264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:57:37.224817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016353+0000","last_change":"2024-04-25T20:56:48.255123+0000","last_active":"2024-04-25T20:59:59.016353+0000","last_peered":"2024-04-25T20:59:59.016353+0000","last_clean":"2024-04-25T20:59:59.016353+0000","last_became_active":"2024-04-25T20:56:48.254680+0000","last_became_peered":"2024-04-25T20:56:48.254680+0000","last_unstale":"2024-04-25T20:59:59.016353+0000","last_undegraded":"2024-04-25T20:59:59.016353+0000","last_fullsized":"2024-04-25T20:59:59.016353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:24.089590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113399+0000","last_change":"2024-04-25T20:57:47.592950+0000","last_active":"2024-04-25T20:59:59.113399+0000","last_peered":"2024-04-25T20:59:59.113399+0000","last_clean":"2024-04-25T20:59:59.113399+0000","last_became_active":"2024-04-25T20:57:47.592486+0000","last_became_peered":"2024-04-25T20:57:47.592486+0000","last_unstale":"2024-04-25T20:59:59.113399+0000","last_undegraded":"2024-04-25T20:59:59.113399+0000","last_fullsized":"2024-04-25T20:59:59.113399+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:43.218061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012933+0000","last_change":"2024-04-25T20:56:48.947508+0000","last_active":"2024-04-25T20:59:59.012933+0000","last_peered":"2024-04-25T20:59:59.012933+0000","last_clean":"2024-04-25T20:59:59.012933+0000","last_became_active":"2024-04-25T20:56:48.946582+0000","last_became_peered":"2024-04-25T20:56:48.946582+0000","last_unstale":"2024-04-25T20:59:59.012933+0000","last_undegraded":"2024-04-25T20:59:59.012933+0000","last_fullsized":"2024-04-25T20:59:59.012933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:11.198667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016523+0000","last_change":"2024-04-25T20:56:51.273560+0000","last_active":"2024-04-25T20:59:59.016523+0000","last_peered":"2024-04-25T20:59:59.016523+0000","last_clean":"2024-04-25T20:59:59.016523+0000","last_became_active":"2024-04-25T20:56:51.273182+0000","last_became_peered":"2024-04-25T20:56:51.273182+0000","last_unstale":"2024-04-25T20:59:59.016523+0000","last_undegraded":"2024-04-25T20:59:59.016523+0000","last_fullsized":"2024-04-25T20:59:59.016523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:47.012771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549107+0000","last_change":"2024-04-25T20:56:48.260754+0000","last_active":"2024-04-25T20:59:59.549107+0000","last_peered":"2024-04-25T20:59:59.549107+0000","last_clean":"2024-04-25T20:59:59.549107+0000","last_became_active":"2024-04-25T20:56:48.260420+0000","last_became_peered":"2024-04-25T20:56:48.260420+0000","last_unstale":"2024-04-25T20:59:59.549107+0000","last_undegraded":"2024-04-25T20:59:59.549107+0000","last_fullsized":"2024-04-25T20:59:59.549107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:00.219023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016420+0000","last_change":"2024-04-25T20:56:51.271944+0000","last_active":"2024-04-25T20:59:59.016420+0000","last_peered":"2024-04-25T20:59:59.016420+0000","last_clean":"2024-04-25T20:59:59.016420+0000","last_became_active":"2024-04-25T20:56:51.270656+0000","last_became_peered":"2024-04-25T20:56:51.270656+0000","last_unstale":"2024-04-25T20:59:59.016420+0000","last_undegraded":"2024-04-25T20:59:59.016420+0000","last_fullsized":"2024-04-25T20:59:59.016420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:00.624888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012713+0000","last_change":"2024-04-25T20:57:47.526172+0000","last_active":"2024-04-25T20:59:59.012713+0000","last_peered":"2024-04-25T20:59:59.012713+0000","last_clean":"2024-04-25T20:59:59.012713+0000","last_became_active":"2024-04-25T20:57:47.525820+0000","last_became_peered":"2024-04-25T20:57:47.525820+0000","last_unstale":"2024-04-25T20:59:59.012713+0000","last_undegraded":"2024-04-25T20:59:59.012713+0000","last_fullsized":"2024-04-25T20:59:59.012713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:56.458959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":211,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:03.047120+0000","last_change":"2024-04-25T20:56:48.946471+0000","last_active":"2024-04-25T21:00:03.047120+0000","last_peered":"2024-04-25T21:00:03.047120+0000","last_clean":"2024-04-25T21:00:03.047120+0000","last_became_active":"2024-04-25T20:56:48.946029+0000","last_became_peered":"2024-04-25T20:56:48.946029+0000","last_unstale":"2024-04-25T21:00:03.047120+0000","last_undegraded":"2024-04-25T21:00:03.047120+0000","last_fullsized":"2024-04-25T21:00:03.047120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:30:16.400157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013018+0000","last_change":"2024-04-25T20:57:48.300758+0000","last_active":"2024-04-25T20:59:59.013018+0000","last_peered":"2024-04-25T20:59:59.013018+0000","last_clean":"2024-04-25T20:59:59.013018+0000","last_became_active":"2024-04-25T20:57:48.300328+0000","last_became_peered":"2024-04-25T20:57:48.300328+0000","last_unstale":"2024-04-25T20:59:59.013018+0000","last_undegraded":"2024-04-25T20:59:59.013018+0000","last_fullsized":"2024-04-25T20:59:59.013018+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:31.931834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041515+0000","last_change":"2024-04-25T20:56:48.258479+0000","last_active":"2024-04-25T20:59:59.041515+0000","last_peered":"2024-04-25T20:59:59.041515+0000","last_clean":"2024-04-25T20:59:59.041515+0000","last_became_active":"2024-04-25T20:56:48.257951+0000","last_became_peered":"2024-04-25T20:56:48.257951+0000","last_unstale":"2024-04-25T20:59:59.041515+0000","last_undegraded":"2024-04-25T20:59:59.041515+0000","last_fullsized":"2024-04-25T20:59:59.041515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:13.812201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":403,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:01.134048+0000","last_change":"2024-04-25T20:55:28.701357+0000","last_active":"2024-04-25T21:00:01.134048+0000","last_peered":"2024-04-25T21:00:01.134048+0000","last_clean":"2024-04-25T21:00:01.134048+0000","last_became_active":"2024-04-25T20:55:28.700906+0000","last_became_peered":"2024-04-25T20:55:28.700906+0000","last_unstale":"2024-04-25T21:00:01.134048+0000","last_undegraded":"2024-04-25T21:00:01.134048+0000","last_fullsized":"2024-04-25T21:00:01.134048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_clean_scrub_stamp":"2024-04-25T20:52:10.334323+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:37:35.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038561+0000","last_change":"2024-04-25T20:56:51.275249+0000","last_active":"2024-04-25T20:59:59.038561+0000","last_peered":"2024-04-25T20:59:59.038561+0000","last_clean":"2024-04-25T20:59:59.038561+0000","last_became_active":"2024-04-25T20:56:51.274938+0000","last_became_peered":"2024-04-25T20:56:51.274938+0000","last_unstale":"2024-04-25T20:59:59.038561+0000","last_undegraded":"2024-04-25T20:59:59.038561+0000","last_fullsized":"2024-04-25T20:59:59.038561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:53.933324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549168+0000","last_change":"2024-04-25T20:56:48.259357+0000","last_active":"2024-04-25T20:59:59.549168+0000","last_peered":"2024-04-25T20:59:59.549168+0000","last_clean":"2024-04-25T20:59:59.549168+0000","last_became_active":"2024-04-25T20:56:48.258840+0000","last_became_peered":"2024-04-25T20:56:48.258840+0000","last_unstale":"2024-04-25T20:59:59.549168+0000","last_undegraded":"2024-04-25T20:59:59.549168+0000","last_fullsized":"2024-04-25T20:59:59.549168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:30.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016354+0000","last_change":"2024-04-25T20:56:48.946498+0000","last_active":"2024-04-25T20:59:59.016354+0000","last_peered":"2024-04-25T20:59:59.016354+0000","last_clean":"2024-04-25T20:59:59.016354+0000","last_became_active":"2024-04-25T20:56:48.946124+0000","last_became_peered":"2024-04-25T20:56:48.946124+0000","last_unstale":"2024-04-25T20:59:59.016354+0000","last_undegraded":"2024-04-25T20:59:59.016354+0000","last_fullsized":"2024-04-25T20:59:59.016354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:08.545574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038986+0000","last_change":"2024-04-25T20:56:51.276093+0000","last_active":"2024-04-25T20:59:59.038986+0000","last_peered":"2024-04-25T20:59:59.038986+0000","last_clean":"2024-04-25T20:59:59.038986+0000","last_became_active":"2024-04-25T20:56:51.275771+0000","last_became_peered":"2024-04-25T20:56:51.275771+0000","last_unstale":"2024-04-25T20:59:59.038986+0000","last_undegraded":"2024-04-25T20:59:59.038986+0000","last_fullsized":"2024-04-25T20:59:59.038986+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:15.159208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'103","reported_seq":236,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:01.628373+0000","last_change":"2024-04-25T20:57:47.533338+0000","last_active":"2024-04-25T21:00:01.628373+0000","last_peered":"2024-04-25T21:00:01.628373+0000","last_clean":"2024-04-25T21:00:01.628373+0000","last_became_active":"2024-04-25T20:57:47.529539+0000","last_became_peered":"2024-04-25T20:57:47.529539+0000","last_unstale":"2024-04-25T21:00:01.628373+0000","last_undegraded":"2024-04-25T21:00:01.628373+0000","last_fullsized":"2024-04-25T21:00:01.628373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:25.099128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":189,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042039+0000","last_change":"2024-04-25T20:56:51.275461+0000","last_active":"2024-04-25T20:59:59.042039+0000","last_peered":"2024-04-25T20:59:59.042039+0000","last_clean":"2024-04-25T20:59:59.042039+0000","last_became_active":"2024-04-25T20:56:51.275087+0000","last_became_peered":"2024-04-25T20:56:51.275087+0000","last_unstale":"2024-04-25T20:59:59.042039+0000","last_undegraded":"2024-04-25T20:59:59.042039+0000","last_fullsized":"2024-04-25T20:59:59.042039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:01:02.514490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041635+0000","last_change":"2024-04-25T20:56:49.031177+0000","last_active":"2024-04-25T20:59:59.041635+0000","last_peered":"2024-04-25T20:59:59.041635+0000","last_clean":"2024-04-25T20:59:59.041635+0000","last_became_active":"2024-04-25T20:56:49.030697+0000","last_became_peered":"2024-04-25T20:56:49.030697+0000","last_unstale":"2024-04-25T20:59:59.041635+0000","last_undegraded":"2024-04-25T20:59:59.041635+0000","last_fullsized":"2024-04-25T20:59:59.041635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:13.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514801+0000","last_change":"2024-04-25T20:57:47.795434+0000","last_active":"2024-04-25T20:59:59.514801+0000","last_peered":"2024-04-25T20:59:59.514801+0000","last_clean":"2024-04-25T20:59:59.514801+0000","last_became_active":"2024-04-25T20:57:47.795036+0000","last_became_peered":"2024-04-25T20:57:47.795036+0000","last_unstale":"2024-04-25T20:59:59.514801+0000","last_undegraded":"2024-04-25T20:59:59.514801+0000","last_fullsized":"2024-04-25T20:59:59.514801+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:48:56.749328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265934+0000","last_change":"2024-04-25T20:56:48.946689+0000","last_active":"2024-04-25T20:59:59.265934+0000","last_peered":"2024-04-25T20:59:59.265934+0000","last_clean":"2024-04-25T20:59:59.265934+0000","last_became_active":"2024-04-25T20:56:48.946229+0000","last_became_peered":"2024-04-25T20:56:48.946229+0000","last_unstale":"2024-04-25T20:59:59.265934+0000","last_undegraded":"2024-04-25T20:59:59.265934+0000","last_fullsized":"2024-04-25T20:59:59.265934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:04.120197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015968+0000","last_change":"2024-04-25T20:56:51.275657+0000","last_active":"2024-04-25T20:59:59.015968+0000","last_peered":"2024-04-25T20:59:59.015968+0000","last_clean":"2024-04-25T20:59:59.015968+0000","last_became_active":"2024-04-25T20:56:51.275237+0000","last_became_peered":"2024-04-25T20:56:51.275237+0000","last_unstale":"2024-04-25T20:59:59.015968+0000","last_undegraded":"2024-04-25T20:59:59.015968+0000","last_fullsized":"2024-04-25T20:59:59.015968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:39.778340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041385+0000","last_change":"2024-04-25T20:56:48.251882+0000","last_active":"2024-04-25T20:59:59.041385+0000","last_peered":"2024-04-25T20:59:59.041385+0000","last_clean":"2024-04-25T20:59:59.041385+0000","last_became_active":"2024-04-25T20:56:48.251571+0000","last_became_peered":"2024-04-25T20:56:48.251571+0000","last_unstale":"2024-04-25T20:59:59.041385+0000","last_undegraded":"2024-04-25T20:59:59.041385+0000","last_fullsized":"2024-04-25T20:59:59.041385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:51.769332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113156+0000","last_change":"2024-04-25T20:56:51.280648+0000","last_active":"2024-04-25T20:59:59.113156+0000","last_peered":"2024-04-25T20:59:59.113156+0000","last_clean":"2024-04-25T20:59:59.113156+0000","last_became_active":"2024-04-25T20:56:51.280232+0000","last_became_peered":"2024-04-25T20:56:51.280232+0000","last_unstale":"2024-04-25T20:59:59.113156+0000","last_undegraded":"2024-04-25T20:59:59.113156+0000","last_fullsized":"2024-04-25T20:59:59.113156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:12.375519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113274+0000","last_change":"2024-04-25T20:57:47.535216+0000","last_active":"2024-04-25T20:59:59.113274+0000","last_peered":"2024-04-25T20:59:59.113274+0000","last_clean":"2024-04-25T20:59:59.113274+0000","last_became_active":"2024-04-25T20:57:47.530348+0000","last_became_peered":"2024-04-25T20:57:47.530348+0000","last_unstale":"2024-04-25T20:59:59.113274+0000","last_undegraded":"2024-04-25T20:59:59.113274+0000","last_fullsized":"2024-04-25T20:59:59.113274+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:01.373541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042321+0000","last_change":"2024-04-25T20:56:51.277166+0000","last_active":"2024-04-25T20:59:59.042321+0000","last_peered":"2024-04-25T20:59:59.042321+0000","last_clean":"2024-04-25T20:59:59.042321+0000","last_became_active":"2024-04-25T20:56:51.276840+0000","last_became_peered":"2024-04-25T20:56:51.276840+0000","last_unstale":"2024-04-25T20:59:59.042321+0000","last_undegraded":"2024-04-25T20:59:59.042321+0000","last_fullsized":"2024-04-25T20:59:59.042321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:52.756660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016118+0000","last_change":"2024-04-25T20:56:48.263426+0000","last_active":"2024-04-25T20:59:59.016118+0000","last_peered":"2024-04-25T20:59:59.016118+0000","last_clean":"2024-04-25T20:59:59.016118+0000","last_became_active":"2024-04-25T20:56:48.262832+0000","last_became_peered":"2024-04-25T20:56:48.262832+0000","last_unstale":"2024-04-25T20:59:59.016118+0000","last_undegraded":"2024-04-25T20:59:59.016118+0000","last_fullsized":"2024-04-25T20:59:59.016118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:09.362161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549131+0000","last_change":"2024-04-25T20:56:51.274267+0000","last_active":"2024-04-25T20:59:59.549131+0000","last_peered":"2024-04-25T20:59:59.549131+0000","last_clean":"2024-04-25T20:59:59.549131+0000","last_became_active":"2024-04-25T20:56:51.273807+0000","last_became_peered":"2024-04-25T20:56:51.273807+0000","last_unstale":"2024-04-25T20:59:59.549131+0000","last_undegraded":"2024-04-25T20:59:59.549131+0000","last_fullsized":"2024-04-25T20:59:59.549131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266148+0000","last_change":"2024-04-25T20:56:49.031011+0000","last_active":"2024-04-25T20:59:59.266148+0000","last_peered":"2024-04-25T20:59:59.266148+0000","last_clean":"2024-04-25T20:59:59.266148+0000","last_became_active":"2024-04-25T20:56:49.030522+0000","last_became_peered":"2024-04-25T20:56:49.030522+0000","last_unstale":"2024-04-25T20:59:59.266148+0000","last_undegraded":"2024-04-25T20:59:59.266148+0000","last_fullsized":"2024-04-25T20:59:59.266148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:00.915454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549501+0000","last_change":"2024-04-25T20:56:51.276989+0000","last_active":"2024-04-25T20:59:59.549501+0000","last_peered":"2024-04-25T20:59:59.549501+0000","last_clean":"2024-04-25T20:59:59.549501+0000","last_became_active":"2024-04-25T20:56:51.275417+0000","last_became_peered":"2024-04-25T20:56:51.275417+0000","last_unstale":"2024-04-25T20:59:59.549501+0000","last_undegraded":"2024-04-25T20:59:59.549501+0000","last_fullsized":"2024-04-25T20:59:59.549501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:27.516062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'65","reported_seq":124,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:01.247407+0000","last_change":"2024-04-25T20:56:48.253713+0000","last_active":"2024-04-25T21:00:01.247407+0000","last_peered":"2024-04-25T21:00:01.247407+0000","last_clean":"2024-04-25T21:00:01.247407+0000","last_became_active":"2024-04-25T20:56:48.253215+0000","last_became_peered":"2024-04-25T20:56:48.253215+0000","last_unstale":"2024-04-25T21:00:01.247407+0000","last_undegraded":"2024-04-25T21:00:01.247407+0000","last_fullsized":"2024-04-25T21:00:01.247407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:57.946840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549273+0000","last_change":"2024-04-25T20:56:51.274691+0000","last_active":"2024-04-25T20:59:59.549273+0000","last_peered":"2024-04-25T20:59:59.549273+0000","last_clean":"2024-04-25T20:59:59.549273+0000","last_became_active":"2024-04-25T20:56:51.274350+0000","last_became_peered":"2024-04-25T20:56:51.274350+0000","last_unstale":"2024-04-25T20:59:59.549273+0000","last_undegraded":"2024-04-25T20:59:59.549273+0000","last_fullsized":"2024-04-25T20:59:59.549273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:52.375409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039075+0000","last_change":"2024-04-25T20:56:48.251616+0000","last_active":"2024-04-25T20:59:59.039075+0000","last_peered":"2024-04-25T20:59:59.039075+0000","last_clean":"2024-04-25T20:59:59.039075+0000","last_became_active":"2024-04-25T20:56:48.251248+0000","last_became_peered":"2024-04-25T20:56:48.251248+0000","last_unstale":"2024-04-25T20:59:59.039075+0000","last_undegraded":"2024-04-25T20:59:59.039075+0000","last_fullsized":"2024-04-25T20:59:59.039075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:51.032356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549388+0000","last_change":"2024-04-25T20:56:51.278372+0000","last_active":"2024-04-25T20:59:59.549388+0000","last_peered":"2024-04-25T20:59:59.549388+0000","last_clean":"2024-04-25T20:59:59.549388+0000","last_became_active":"2024-04-25T20:56:51.277989+0000","last_became_peered":"2024-04-25T20:56:51.277989+0000","last_unstale":"2024-04-25T20:59:59.549388+0000","last_undegraded":"2024-04-25T20:59:59.549388+0000","last_fullsized":"2024-04-25T20:59:59.549388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.017713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113230+0000","last_change":"2024-04-25T20:56:51.276270+0000","last_active":"2024-04-25T20:59:59.113230+0000","last_peered":"2024-04-25T20:59:59.113230+0000","last_clean":"2024-04-25T20:59:59.113230+0000","last_became_active":"2024-04-25T20:56:51.275948+0000","last_became_peered":"2024-04-25T20:56:51.275948+0000","last_unstale":"2024-04-25T20:59:59.113230+0000","last_undegraded":"2024-04-25T20:59:59.113230+0000","last_fullsized":"2024-04-25T20:59:59.113230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:33.548439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265845+0000","last_change":"2024-04-25T20:56:51.283931+0000","last_active":"2024-04-25T20:59:59.265845+0000","last_peered":"2024-04-25T20:59:59.265845+0000","last_clean":"2024-04-25T20:59:59.265845+0000","last_became_active":"2024-04-25T20:56:51.283120+0000","last_became_peered":"2024-04-25T20:56:51.283120+0000","last_unstale":"2024-04-25T20:59:59.265845+0000","last_undegraded":"2024-04-25T20:59:59.265845+0000","last_fullsized":"2024-04-25T20:59:59.265845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:26.676750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265827+0000","last_change":"2024-04-25T20:56:51.286013+0000","last_active":"2024-04-25T20:59:59.265827+0000","last_peered":"2024-04-25T20:59:59.265827+0000","last_clean":"2024-04-25T20:59:59.265827+0000","last_became_active":"2024-04-25T20:56:51.285645+0000","last_became_peered":"2024-04-25T20:56:51.285645+0000","last_unstale":"2024-04-25T20:59:59.265827+0000","last_undegraded":"2024-04-25T20:59:59.265827+0000","last_fullsized":"2024-04-25T20:59:59.265827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:51.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514609+0000","last_change":"2024-04-25T20:56:51.272567+0000","last_active":"2024-04-25T20:59:59.514609+0000","last_peered":"2024-04-25T20:59:59.514609+0000","last_clean":"2024-04-25T20:59:59.514609+0000","last_became_active":"2024-04-25T20:56:51.271427+0000","last_became_peered":"2024-04-25T20:56:51.271427+0000","last_unstale":"2024-04-25T20:59:59.514609+0000","last_undegraded":"2024-04-25T20:59:59.514609+0000","last_fullsized":"2024-04-25T20:59:59.514609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:19.945982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038896+0000","last_change":"2024-04-25T20:56:51.272448+0000","last_active":"2024-04-25T20:59:59.038896+0000","last_peered":"2024-04-25T20:59:59.038896+0000","last_clean":"2024-04-25T20:59:59.038896+0000","last_became_active":"2024-04-25T20:56:51.271965+0000","last_became_peered":"2024-04-25T20:56:51.271965+0000","last_unstale":"2024-04-25T20:59:59.038896+0000","last_undegraded":"2024-04-25T20:59:59.038896+0000","last_fullsized":"2024-04-25T20:59:59.038896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:03.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113416+0000","last_change":"2024-04-25T20:56:51.280260+0000","last_active":"2024-04-25T20:59:59.113416+0000","last_peered":"2024-04-25T20:59:59.113416+0000","last_clean":"2024-04-25T20:59:59.113416+0000","last_became_active":"2024-04-25T20:56:51.279864+0000","last_became_peered":"2024-04-25T20:56:51.279864+0000","last_unstale":"2024-04-25T20:59:59.113416+0000","last_undegraded":"2024-04-25T20:59:59.113416+0000","last_fullsized":"2024-04-25T20:59:59.113416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.136824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266115+0000","last_change":"2024-04-25T20:56:51.279041+0000","last_active":"2024-04-25T20:59:59.266115+0000","last_peered":"2024-04-25T20:59:59.266115+0000","last_clean":"2024-04-25T20:59:59.266115+0000","last_became_active":"2024-04-25T20:56:51.278302+0000","last_became_peered":"2024-04-25T20:56:51.278302+0000","last_unstale":"2024-04-25T20:59:59.266115+0000","last_undegraded":"2024-04-25T20:59:59.266115+0000","last_fullsized":"2024-04-25T20:59:59.266115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:02.510919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041907+0000","last_change":"2024-04-25T20:56:51.275618+0000","last_active":"2024-04-25T20:59:59.041907+0000","last_peered":"2024-04-25T20:59:59.041907+0000","last_clean":"2024-04-25T20:59:59.041907+0000","last_became_active":"2024-04-25T20:56:51.275249+0000","last_became_peered":"2024-04-25T20:56:51.275249+0000","last_unstale":"2024-04-25T20:59:59.041907+0000","last_undegraded":"2024-04-25T20:59:59.041907+0000","last_fullsized":"2024-04-25T20:59:59.041907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:01.793083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119251+0000","last_change":"2024-04-25T20:56:51.275868+0000","last_active":"2024-04-25T20:59:59.119251+0000","last_peered":"2024-04-25T20:59:59.119251+0000","last_clean":"2024-04-25T20:59:59.119251+0000","last_became_active":"2024-04-25T20:56:51.275544+0000","last_became_peered":"2024-04-25T20:56:51.275544+0000","last_unstale":"2024-04-25T20:59:59.119251+0000","last_undegraded":"2024-04-25T20:59:59.119251+0000","last_fullsized":"2024-04-25T20:59:59.119251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:31.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041380+0000","last_change":"2024-04-25T20:56:51.274963+0000","last_active":"2024-04-25T20:59:59.041380+0000","last_peered":"2024-04-25T20:59:59.041380+0000","last_clean":"2024-04-25T20:59:59.041380+0000","last_became_active":"2024-04-25T20:56:51.274597+0000","last_became_peered":"2024-04-25T20:56:51.274597+0000","last_unstale":"2024-04-25T20:59:59.041380+0000","last_undegraded":"2024-04-25T20:59:59.041380+0000","last_fullsized":"2024-04-25T20:59:59.041380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:14.464560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266007+0000","last_change":"2024-04-25T20:56:51.279602+0000","last_active":"2024-04-25T20:59:59.266007+0000","last_peered":"2024-04-25T20:59:59.266007+0000","last_clean":"2024-04-25T20:59:59.266007+0000","last_became_active":"2024-04-25T20:56:51.279221+0000","last_became_peered":"2024-04-25T20:56:51.279221+0000","last_unstale":"2024-04-25T20:59:59.266007+0000","last_undegraded":"2024-04-25T20:59:59.266007+0000","last_fullsized":"2024-04-25T20:59:59.266007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:26.960492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039002+0000","last_change":"2024-04-25T20:56:51.280465+0000","last_active":"2024-04-25T20:59:59.039002+0000","last_peered":"2024-04-25T20:59:59.039002+0000","last_clean":"2024-04-25T20:59:59.039002+0000","last_became_active":"2024-04-25T20:56:51.280168+0000","last_became_peered":"2024-04-25T20:56:51.280168+0000","last_unstale":"2024-04-25T20:59:59.039002+0000","last_undegraded":"2024-04-25T20:59:59.039002+0000","last_fullsized":"2024-04-25T20:59:59.039002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.211113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113237+0000","last_change":"2024-04-25T20:56:51.283084+0000","last_active":"2024-04-25T20:59:59.113237+0000","last_peered":"2024-04-25T20:59:59.113237+0000","last_clean":"2024-04-25T20:59:59.113237+0000","last_became_active":"2024-04-25T20:56:51.282692+0000","last_became_peered":"2024-04-25T20:56:51.282692+0000","last_unstale":"2024-04-25T20:59:59.113237+0000","last_undegraded":"2024-04-25T20:59:59.113237+0000","last_fullsized":"2024-04-25T20:59:59.113237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:13:27.527112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796400,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1444,"num_read_kb":939,"num_write":423,"num_write_kb":2357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2389200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1409024,"data_stored":886419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37460,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707692,"statfs":{"total":95995035648,"available":95956676608,"internally_reserved":0,"allocated":2052096,"data_stored":1483726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776192,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1425408,"data_stored":886655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037829,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":493359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1626112,"data_stored":1086270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1019904,"data_stored":492828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":562591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":494099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":564473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28856,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716296,"statfs":{"total":95995035648,"available":95965487104,"internally_reserved":0,"allocated":1761280,"data_stored":1214587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920856,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":494105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724628,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33220,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711932,"statfs":{"total":95995035648,"available":95961018368,"internally_reserved":0,"allocated":1970176,"data_stored":1413346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":400620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:00:07.908 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T21:00:07.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:00:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:08 smithi047 ceph-mon[29195]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-25T21:00:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:08 smithi047 ceph-mon[29195]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:08.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:08 smithi186 ceph-mon[32529]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-25T21:00:08.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:08 smithi186 ceph-mon[32529]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:09.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:08 smithi092 ceph-mon[32524]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-25T21:00:09.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:08 smithi092 ceph-mon[32524]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 -- 172.21.15.47:0/3414883254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 msgr2=0x7f93f806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 --2- 172.21.15.47:0/3414883254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 0x7f93f806b440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009990 tx=0x7f93e8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 -- 172.21.15.47:0/3414883254 shutdown_connections 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 --2- 172.21.15.47:0/3414883254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f806b980 0x7f93f810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 --2- 172.21.15.47:0/3414883254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93f8108a20 0x7f93f8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 --2- 172.21.15.47:0/3414883254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 0x7f93f806b440 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 -- 172.21.15.47:0/3414883254 >> 172.21.15.47:0/3414883254 conn(0x7f93f80788b0 msgr2=0x7f93f8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 -- 172.21.15.47:0/3414883254 shutdown_connections 2024-04-25T21:00:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.859+0000 7f9400036700 1 -- 172.21.15.47:0/3414883254 wait complete. 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f9400036700 1 Processor -- start 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f9400036700 1 -- start start 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f9400036700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 0x7f93f819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f9400036700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93f806b980 0x7f93f819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f9400036700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f8108a20 0x7f93f819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f9400036700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112be0 con 0x7f93f806afe0 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f9400036700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112a60 con 0x7f93f806b980 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f9400036700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112d60 con 0x7f93f8108a20 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fddd2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 0x7f93f819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fddd2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 0x7f93f819b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42280/0 (socket says 172.21.15.47:42280) 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fddd2700 1 -- 172.21.15.47:0/1153564117 learned_addr learned my addr 172.21.15.47:0/1153564117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fd5d1700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93f806b980 0x7f93f819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fe5d3700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f8108a20 0x7f93f819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fddd2700 1 -- 172.21.15.47:0/1153564117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f8108a20 msgr2=0x7f93f819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fddd2700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f8108a20 0x7f93f819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fddd2700 1 -- 172.21.15.47:0/1153564117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93f806b980 msgr2=0x7f93f819b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fddd2700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93f806b980 0x7f93f819b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.860+0000 7f93fddd2700 1 -- 172.21.15.47:0/1153564117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f81a0300 con 0x7f93f806afe0 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.861+0000 7f93fddd2700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 0x7f93f819b2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c920 tx=0x7f93f400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.861+0000 7f93eeffd700 1 -- 172.21.15.47:0/1153564117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4004a30 con 0x7f93f806afe0 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.861+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f81a05f0 con 0x7f93f806afe0 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.861+0000 7f93eeffd700 1 -- 172.21.15.47:0/1153564117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f40054d0 con 0x7f93f806afe0 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.861+0000 7f93eeffd700 1 -- 172.21.15.47:0/1153564117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4007c70 con 0x7f93f806afe0 2024-04-25T21:00:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.861+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a7e70 con 0x7f93f806afe0 2024-04-25T21:00:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.862+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f8069730 con 0x7f93f806afe0 2024-04-25T21:00:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.863+0000 7f93eeffd700 1 -- 172.21.15.47:0/1153564117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f93f4004c60 con 0x7f93f806afe0 2024-04-25T21:00:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.864+0000 7f93eeffd700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f93e40a3f50 0x7f93e40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.864+0000 7f93fd5d1700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f93e40a3f50 0x7f93e40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.864+0000 7f93eeffd700 1 -- 172.21.15.47:0/1153564117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f93f40d0a00 con 0x7f93f806afe0 2024-04-25T21:00:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.864+0000 7f93fd5d1700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f93e40a3f50 0x7f93e40a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b5c0 tx=0x7f93e8005960 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:09.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:09.867+0000 7f93eeffd700 1 -- 172.21.15.47:0/1153564117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93f4014030 con 0x7f93f806afe0 2024-04-25T21:00:10.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.071+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93f8062e80 con 0x7f93e40a3f50 2024-04-25T21:00:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.078+0000 7f93eeffd700 1 -- 172.21.15.47:0/1153564117 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257767 (secure 0 0 0) 0x7f93f8062e80 con 0x7f93e40a3f50 2024-04-25T21:00:10.082 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f93e40a3f50 msgr2=0x7f93e40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f93e40a3f50 0x7f93e40a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b5c0 tx=0x7f93e8005960 comp rx=0 tx=0).stop 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 msgr2=0x7f93f819b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 0x7f93f819b2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c920 tx=0x7f93f400cc30 comp rx=0 tx=0).stop 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 shutdown_connections 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f93e40a3f50 0x7f93e40a6410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f806afe0 0x7f93f819b2f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f8108a20 0x7f93f819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 --2- 172.21.15.47:0/1153564117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93f806b980 0x7f93f819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 >> 172.21.15.47:0/1153564117 conn(0x7f93f80788b0 msgr2=0x7f93f810de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 shutdown_connections 2024-04-25T21:00:10.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:10.081+0000 7f9400036700 1 -- 172.21.15.47:0/1153564117 wait complete. 2024-04-25T21:00:10.085 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-25T21:00:10.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:10 smithi047 ceph-mon[29195]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-25T21:00:10.736 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-25T21:00:09.030089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387002,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1681,"num_read_kb":1269,"num_write":626,"num_write_kb":5297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":961,"ondisk_log_size":961,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365716,"kb_used_data":16176,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124576108,"statfs":{"total":1151940427776,"available":1151565934592,"internally_reserved":0,"allocated":16564224,"data_stored":10072458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199543,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":386,"num_read_kb":460,"num_write":150,"num_write_kb":919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.994091"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113432+0000","last_change":"2024-04-25T20:56:51.283722+0000","last_active":"2024-04-25T20:59:59.113432+0000","last_peered":"2024-04-25T20:59:59.113432+0000","last_clean":"2024-04-25T20:59:59.113432+0000","last_became_active":"2024-04-25T20:56:51.283359+0000","last_became_peered":"2024-04-25T20:56:51.283359+0000","last_unstale":"2024-04-25T20:59:59.113432+0000","last_undegraded":"2024-04-25T20:59:59.113432+0000","last_fullsized":"2024-04-25T20:59:59.113432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:47.314687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514767+0000","last_change":"2024-04-25T20:56:51.273199+0000","last_active":"2024-04-25T20:59:59.514767+0000","last_peered":"2024-04-25T20:59:59.514767+0000","last_clean":"2024-04-25T20:59:59.514767+0000","last_became_active":"2024-04-25T20:56:51.272838+0000","last_became_peered":"2024-04-25T20:56:51.272838+0000","last_unstale":"2024-04-25T20:59:59.514767+0000","last_undegraded":"2024-04-25T20:59:59.514767+0000","last_fullsized":"2024-04-25T20:59:59.514767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:58.033315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514639+0000","last_change":"2024-04-25T20:56:51.281461+0000","last_active":"2024-04-25T20:59:59.514639+0000","last_peered":"2024-04-25T20:59:59.514639+0000","last_clean":"2024-04-25T20:59:59.514639+0000","last_became_active":"2024-04-25T20:56:51.281141+0000","last_became_peered":"2024-04-25T20:56:51.281141+0000","last_unstale":"2024-04-25T20:59:59.514639+0000","last_undegraded":"2024-04-25T20:59:59.514639+0000","last_fullsized":"2024-04-25T20:59:59.514639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041716+0000","last_change":"2024-04-25T20:56:51.283187+0000","last_active":"2024-04-25T20:59:59.041716+0000","last_peered":"2024-04-25T20:59:59.041716+0000","last_clean":"2024-04-25T20:59:59.041716+0000","last_became_active":"2024-04-25T20:56:51.282640+0000","last_became_peered":"2024-04-25T20:56:51.282640+0000","last_unstale":"2024-04-25T20:59:59.041716+0000","last_undegraded":"2024-04-25T20:59:59.041716+0000","last_fullsized":"2024-04-25T20:59:59.041716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:15.588694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016083+0000","last_change":"2024-04-25T20:56:51.273204+0000","last_active":"2024-04-25T20:59:59.016083+0000","last_peered":"2024-04-25T20:59:59.016083+0000","last_clean":"2024-04-25T20:59:59.016083+0000","last_became_active":"2024-04-25T20:56:51.272126+0000","last_became_peered":"2024-04-25T20:56:51.272126+0000","last_unstale":"2024-04-25T20:59:59.016083+0000","last_undegraded":"2024-04-25T20:59:59.016083+0000","last_fullsized":"2024-04-25T20:59:59.016083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:31.684294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016332+0000","last_change":"2024-04-25T20:56:51.270762+0000","last_active":"2024-04-25T20:59:59.016332+0000","last_peered":"2024-04-25T20:59:59.016332+0000","last_clean":"2024-04-25T20:59:59.016332+0000","last_became_active":"2024-04-25T20:56:51.270444+0000","last_became_peered":"2024-04-25T20:56:51.270444+0000","last_unstale":"2024-04-25T20:59:59.016332+0000","last_undegraded":"2024-04-25T20:59:59.016332+0000","last_fullsized":"2024-04-25T20:59:59.016332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:00.155417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549002+0000","last_change":"2024-04-25T20:56:51.276029+0000","last_active":"2024-04-25T20:59:59.549002+0000","last_peered":"2024-04-25T20:59:59.549002+0000","last_clean":"2024-04-25T20:59:59.549002+0000","last_became_active":"2024-04-25T20:56:51.275272+0000","last_became_peered":"2024-04-25T20:56:51.275272+0000","last_unstale":"2024-04-25T20:59:59.549002+0000","last_undegraded":"2024-04-25T20:59:59.549002+0000","last_fullsized":"2024-04-25T20:59:59.549002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.428444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041346+0000","last_change":"2024-04-25T20:56:51.273769+0000","last_active":"2024-04-25T20:59:59.041346+0000","last_peered":"2024-04-25T20:59:59.041346+0000","last_clean":"2024-04-25T20:59:59.041346+0000","last_became_active":"2024-04-25T20:56:51.273412+0000","last_became_peered":"2024-04-25T20:56:51.273412+0000","last_unstale":"2024-04-25T20:59:59.041346+0000","last_undegraded":"2024-04-25T20:59:59.041346+0000","last_fullsized":"2024-04-25T20:59:59.041346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.324332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266500+0000","last_change":"2024-04-25T20:56:51.277097+0000","last_active":"2024-04-25T20:59:59.266500+0000","last_peered":"2024-04-25T20:59:59.266500+0000","last_clean":"2024-04-25T20:59:59.266500+0000","last_became_active":"2024-04-25T20:56:51.276768+0000","last_became_peered":"2024-04-25T20:56:51.276768+0000","last_unstale":"2024-04-25T20:59:59.266500+0000","last_undegraded":"2024-04-25T20:59:59.266500+0000","last_fullsized":"2024-04-25T20:59:59.266500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:23.276470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016837+0000","last_change":"2024-04-25T20:56:51.279046+0000","last_active":"2024-04-25T20:59:59.016837+0000","last_peered":"2024-04-25T20:59:59.016837+0000","last_clean":"2024-04-25T20:59:59.016837+0000","last_became_active":"2024-04-25T20:56:51.278701+0000","last_became_peered":"2024-04-25T20:56:51.278701+0000","last_unstale":"2024-04-25T20:59:59.016837+0000","last_undegraded":"2024-04-25T20:59:59.016837+0000","last_fullsized":"2024-04-25T20:59:59.016837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.370482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549375+0000","last_change":"2024-04-25T20:56:51.271593+0000","last_active":"2024-04-25T20:59:59.549375+0000","last_peered":"2024-04-25T20:59:59.549375+0000","last_clean":"2024-04-25T20:59:59.549375+0000","last_became_active":"2024-04-25T20:56:51.271224+0000","last_became_peered":"2024-04-25T20:56:51.271224+0000","last_unstale":"2024-04-25T20:59:59.549375+0000","last_undegraded":"2024-04-25T20:59:59.549375+0000","last_fullsized":"2024-04-25T20:59:59.549375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:46.847929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041553+0000","last_change":"2024-04-25T20:56:51.278281+0000","last_active":"2024-04-25T20:59:59.041553+0000","last_peered":"2024-04-25T20:59:59.041553+0000","last_clean":"2024-04-25T20:59:59.041553+0000","last_became_active":"2024-04-25T20:56:51.277947+0000","last_became_peered":"2024-04-25T20:56:51.277947+0000","last_unstale":"2024-04-25T20:59:59.041553+0000","last_undegraded":"2024-04-25T20:59:59.041553+0000","last_fullsized":"2024-04-25T20:59:59.041553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:02.509296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039058+0000","last_change":"2024-04-25T20:56:51.273784+0000","last_active":"2024-04-25T20:59:59.039058+0000","last_peered":"2024-04-25T20:59:59.039058+0000","last_clean":"2024-04-25T20:59:59.039058+0000","last_became_active":"2024-04-25T20:56:51.273489+0000","last_became_peered":"2024-04-25T20:56:51.273489+0000","last_unstale":"2024-04-25T20:59:59.039058+0000","last_undegraded":"2024-04-25T20:59:59.039058+0000","last_fullsized":"2024-04-25T20:59:59.039058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:00.765186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514418+0000","last_change":"2024-04-25T20:56:51.269829+0000","last_active":"2024-04-25T20:59:59.514418+0000","last_peered":"2024-04-25T20:59:59.514418+0000","last_clean":"2024-04-25T20:59:59.514418+0000","last_became_active":"2024-04-25T20:56:51.269446+0000","last_became_peered":"2024-04-25T20:56:51.269446+0000","last_unstale":"2024-04-25T20:59:59.514418+0000","last_undegraded":"2024-04-25T20:59:59.514418+0000","last_fullsized":"2024-04-25T20:59:59.514418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:12.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514335+0000","last_change":"2024-04-25T20:56:51.278117+0000","last_active":"2024-04-25T20:59:59.514335+0000","last_peered":"2024-04-25T20:59:59.514335+0000","last_clean":"2024-04-25T20:59:59.514335+0000","last_became_active":"2024-04-25T20:56:51.277596+0000","last_became_peered":"2024-04-25T20:56:51.277596+0000","last_unstale":"2024-04-25T20:59:59.514335+0000","last_undegraded":"2024-04-25T20:59:59.514335+0000","last_fullsized":"2024-04-25T20:59:59.514335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:28.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265776+0000","last_change":"2024-04-25T20:56:51.281575+0000","last_active":"2024-04-25T20:59:59.265776+0000","last_peered":"2024-04-25T20:59:59.265776+0000","last_clean":"2024-04-25T20:59:59.265776+0000","last_became_active":"2024-04-25T20:56:51.281217+0000","last_became_peered":"2024-04-25T20:56:51.281217+0000","last_unstale":"2024-04-25T20:59:59.265776+0000","last_undegraded":"2024-04-25T20:59:59.265776+0000","last_fullsized":"2024-04-25T20:59:59.265776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:05.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113547+0000","last_change":"2024-04-25T20:56:51.284608+0000","last_active":"2024-04-25T20:59:59.113547+0000","last_peered":"2024-04-25T20:59:59.113547+0000","last_clean":"2024-04-25T20:59:59.113547+0000","last_became_active":"2024-04-25T20:56:51.284257+0000","last_became_peered":"2024-04-25T20:56:51.284257+0000","last_unstale":"2024-04-25T20:59:59.113547+0000","last_undegraded":"2024-04-25T20:59:59.113547+0000","last_fullsized":"2024-04-25T20:59:59.113547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:44:59.184192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013207+0000","last_change":"2024-04-25T20:56:51.276770+0000","last_active":"2024-04-25T20:59:59.013207+0000","last_peered":"2024-04-25T20:59:59.013207+0000","last_clean":"2024-04-25T20:59:59.013207+0000","last_became_active":"2024-04-25T20:56:51.276431+0000","last_became_peered":"2024-04-25T20:56:51.276431+0000","last_unstale":"2024-04-25T20:59:59.013207+0000","last_undegraded":"2024-04-25T20:59:59.013207+0000","last_fullsized":"2024-04-25T20:59:59.013207+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:14:25.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265999+0000","last_change":"2024-04-25T20:56:51.283728+0000","last_active":"2024-04-25T20:59:59.265999+0000","last_peered":"2024-04-25T20:59:59.265999+0000","last_clean":"2024-04-25T20:59:59.265999+0000","last_became_active":"2024-04-25T20:56:51.283370+0000","last_became_peered":"2024-04-25T20:56:51.283370+0000","last_unstale":"2024-04-25T20:59:59.265999+0000","last_undegraded":"2024-04-25T20:59:59.265999+0000","last_fullsized":"2024-04-25T20:59:59.265999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:18.042769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012950+0000","last_change":"2024-04-25T20:56:48.259737+0000","last_active":"2024-04-25T20:59:59.012950+0000","last_peered":"2024-04-25T20:59:59.012950+0000","last_clean":"2024-04-25T20:59:59.012950+0000","last_became_active":"2024-04-25T20:56:48.259442+0000","last_became_peered":"2024-04-25T20:56:48.259442+0000","last_unstale":"2024-04-25T20:59:59.012950+0000","last_undegraded":"2024-04-25T20:59:59.012950+0000","last_fullsized":"2024-04-25T20:59:59.012950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:23.191536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042189+0000","last_change":"2024-04-25T20:56:51.274510+0000","last_active":"2024-04-25T20:59:59.042189+0000","last_peered":"2024-04-25T20:59:59.042189+0000","last_clean":"2024-04-25T20:59:59.042189+0000","last_became_active":"2024-04-25T20:56:51.274137+0000","last_became_peered":"2024-04-25T20:56:51.274137+0000","last_unstale":"2024-04-25T20:59:59.042189+0000","last_undegraded":"2024-04-25T20:59:59.042189+0000","last_fullsized":"2024-04-25T20:59:59.042189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:05.903617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042405+0000","last_change":"2024-04-25T20:56:48.947850+0000","last_active":"2024-04-25T20:59:59.042405+0000","last_peered":"2024-04-25T20:59:59.042405+0000","last_clean":"2024-04-25T20:59:59.042405+0000","last_became_active":"2024-04-25T20:56:48.946160+0000","last_became_peered":"2024-04-25T20:56:48.946160+0000","last_unstale":"2024-04-25T20:59:59.042405+0000","last_undegraded":"2024-04-25T20:59:59.042405+0000","last_fullsized":"2024-04-25T20:59:59.042405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:32.692390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.515020+0000","last_change":"2024-04-25T20:56:51.280478+0000","last_active":"2024-04-25T20:59:59.515020+0000","last_peered":"2024-04-25T20:59:59.515020+0000","last_clean":"2024-04-25T20:59:59.515020+0000","last_became_active":"2024-04-25T20:56:51.280173+0000","last_became_peered":"2024-04-25T20:56:51.280173+0000","last_unstale":"2024-04-25T20:59:59.515020+0000","last_undegraded":"2024-04-25T20:59:59.515020+0000","last_fullsized":"2024-04-25T20:59:59.515020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:48.838885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016506+0000","last_change":"2024-04-25T20:57:47.535264+0000","last_active":"2024-04-25T20:59:59.016506+0000","last_peered":"2024-04-25T20:59:59.016506+0000","last_clean":"2024-04-25T20:59:59.016506+0000","last_became_active":"2024-04-25T20:57:47.531152+0000","last_became_peered":"2024-04-25T20:57:47.531152+0000","last_unstale":"2024-04-25T20:59:59.016506+0000","last_undegraded":"2024-04-25T20:59:59.016506+0000","last_fullsized":"2024-04-25T20:59:59.016506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:19.085455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265973+0000","last_change":"2024-04-25T20:56:51.278383+0000","last_active":"2024-04-25T20:59:59.265973+0000","last_peered":"2024-04-25T20:59:59.265973+0000","last_clean":"2024-04-25T20:59:59.265973+0000","last_became_active":"2024-04-25T20:56:51.277603+0000","last_became_peered":"2024-04-25T20:56:51.277603+0000","last_unstale":"2024-04-25T20:59:59.265973+0000","last_undegraded":"2024-04-25T20:59:59.265973+0000","last_fullsized":"2024-04-25T20:59:59.265973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:12.197940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016420+0000","last_change":"2024-04-25T20:56:48.258869+0000","last_active":"2024-04-25T20:59:59.016420+0000","last_peered":"2024-04-25T20:59:59.016420+0000","last_clean":"2024-04-25T20:59:59.016420+0000","last_became_active":"2024-04-25T20:56:48.258414+0000","last_became_peered":"2024-04-25T20:56:48.258414+0000","last_unstale":"2024-04-25T20:59:59.016420+0000","last_undegraded":"2024-04-25T20:59:59.016420+0000","last_fullsized":"2024-04-25T20:59:59.016420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:09.889936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013004+0000","last_change":"2024-04-25T20:57:48.300637+0000","last_active":"2024-04-25T20:59:59.013004+0000","last_peered":"2024-04-25T20:59:59.013004+0000","last_clean":"2024-04-25T20:59:59.013004+0000","last_became_active":"2024-04-25T20:57:48.300285+0000","last_became_peered":"2024-04-25T20:57:48.300285+0000","last_unstale":"2024-04-25T20:59:59.013004+0000","last_undegraded":"2024-04-25T20:59:59.013004+0000","last_fullsized":"2024-04-25T20:59:59.013004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:31.810671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:04.017318+0000","last_change":"2024-04-25T20:56:48.262543+0000","last_active":"2024-04-25T21:00:04.017318+0000","last_peered":"2024-04-25T21:00:04.017318+0000","last_clean":"2024-04-25T21:00:04.017318+0000","last_became_active":"2024-04-25T20:56:48.262086+0000","last_became_peered":"2024-04-25T20:56:48.262086+0000","last_unstale":"2024-04-25T21:00:04.017318+0000","last_undegraded":"2024-04-25T21:00:04.017318+0000","last_fullsized":"2024-04-25T21:00:04.017318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:43.722131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015962+0000","last_change":"2024-04-25T20:56:51.273500+0000","last_active":"2024-04-25T20:59:59.015962+0000","last_peered":"2024-04-25T20:59:59.015962+0000","last_clean":"2024-04-25T20:59:59.015962+0000","last_became_active":"2024-04-25T20:56:51.272882+0000","last_became_peered":"2024-04-25T20:56:51.272882+0000","last_unstale":"2024-04-25T20:59:59.015962+0000","last_undegraded":"2024-04-25T20:59:59.015962+0000","last_fullsized":"2024-04-25T20:59:59.015962+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:05.682258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016218+0000","last_change":"2024-04-25T20:56:48.255225+0000","last_active":"2024-04-25T20:59:59.016218+0000","last_peered":"2024-04-25T20:59:59.016218+0000","last_clean":"2024-04-25T20:59:59.016218+0000","last_became_active":"2024-04-25T20:56:48.254644+0000","last_became_peered":"2024-04-25T20:56:48.254644+0000","last_unstale":"2024-04-25T20:59:59.016218+0000","last_undegraded":"2024-04-25T20:59:59.016218+0000","last_fullsized":"2024-04-25T20:59:59.016218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:35.724612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119382+0000","last_change":"2024-04-25T20:56:51.270709+0000","last_active":"2024-04-25T20:59:59.119382+0000","last_peered":"2024-04-25T20:59:59.119382+0000","last_clean":"2024-04-25T20:59:59.119382+0000","last_became_active":"2024-04-25T20:56:51.270295+0000","last_became_peered":"2024-04-25T20:56:51.270295+0000","last_unstale":"2024-04-25T20:59:59.119382+0000","last_undegraded":"2024-04-25T20:59:59.119382+0000","last_fullsized":"2024-04-25T20:59:59.119382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:41.998953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038784+0000","last_change":"2024-04-25T20:56:48.256498+0000","last_active":"2024-04-25T20:59:59.038784+0000","last_peered":"2024-04-25T20:59:59.038784+0000","last_clean":"2024-04-25T20:59:59.038784+0000","last_became_active":"2024-04-25T20:56:48.256140+0000","last_became_peered":"2024-04-25T20:56:48.256140+0000","last_unstale":"2024-04-25T20:59:59.038784+0000","last_undegraded":"2024-04-25T20:59:59.038784+0000","last_fullsized":"2024-04-25T20:59:59.038784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:41.458235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041426+0000","last_change":"2024-04-25T20:56:51.267575+0000","last_active":"2024-04-25T20:59:59.041426+0000","last_peered":"2024-04-25T20:59:59.041426+0000","last_clean":"2024-04-25T20:59:59.041426+0000","last_became_active":"2024-04-25T20:56:51.267265+0000","last_became_peered":"2024-04-25T20:56:51.267265+0000","last_unstale":"2024-04-25T20:59:59.041426+0000","last_undegraded":"2024-04-25T20:59:59.041426+0000","last_fullsized":"2024-04-25T20:59:59.041426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:30.760555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119218+0000","last_change":"2024-04-25T20:56:48.260747+0000","last_active":"2024-04-25T20:59:59.119218+0000","last_peered":"2024-04-25T20:59:59.119218+0000","last_clean":"2024-04-25T20:59:59.119218+0000","last_became_active":"2024-04-25T20:56:48.260339+0000","last_became_peered":"2024-04-25T20:56:48.260339+0000","last_unstale":"2024-04-25T20:59:59.119218+0000","last_undegraded":"2024-04-25T20:59:59.119218+0000","last_fullsized":"2024-04-25T20:59:59.119218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:19.769631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012748+0000","last_change":"2024-04-25T20:57:47.795125+0000","last_active":"2024-04-25T20:59:59.012748+0000","last_peered":"2024-04-25T20:59:59.012748+0000","last_clean":"2024-04-25T20:59:59.012748+0000","last_became_active":"2024-04-25T20:57:47.794642+0000","last_became_peered":"2024-04-25T20:57:47.794642+0000","last_unstale":"2024-04-25T20:59:59.012748+0000","last_undegraded":"2024-04-25T20:59:59.012748+0000","last_fullsized":"2024-04-25T20:59:59.012748+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:15.443731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":210,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:04.016806+0000","last_change":"2024-04-25T20:56:48.256113+0000","last_active":"2024-04-25T21:00:04.016806+0000","last_peered":"2024-04-25T21:00:04.016806+0000","last_clean":"2024-04-25T21:00:04.016806+0000","last_became_active":"2024-04-25T20:56:48.255644+0000","last_became_peered":"2024-04-25T20:56:48.255644+0000","last_unstale":"2024-04-25T21:00:04.016806+0000","last_undegraded":"2024-04-25T21:00:04.016806+0000","last_fullsized":"2024-04-25T21:00:04.016806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:12.100099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":177,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266407+0000","last_change":"2024-04-25T20:56:51.276990+0000","last_active":"2024-04-25T20:59:59.266407+0000","last_peered":"2024-04-25T20:59:59.266407+0000","last_clean":"2024-04-25T20:59:59.266407+0000","last_became_active":"2024-04-25T20:56:51.276638+0000","last_became_peered":"2024-04-25T20:56:51.276638+0000","last_unstale":"2024-04-25T20:59:59.266407+0000","last_undegraded":"2024-04-25T20:59:59.266407+0000","last_fullsized":"2024-04-25T20:59:59.266407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:06.645278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549502+0000","last_change":"2024-04-25T20:56:48.263216+0000","last_active":"2024-04-25T20:59:59.549502+0000","last_peered":"2024-04-25T20:59:59.549502+0000","last_clean":"2024-04-25T20:59:59.549502+0000","last_became_active":"2024-04-25T20:56:48.262854+0000","last_became_peered":"2024-04-25T20:56:48.262854+0000","last_unstale":"2024-04-25T20:59:59.549502+0000","last_undegraded":"2024-04-25T20:59:59.549502+0000","last_fullsized":"2024-04-25T20:59:59.549502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:00.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514957+0000","last_change":"2024-04-25T20:56:51.278479+0000","last_active":"2024-04-25T20:59:59.514957+0000","last_peered":"2024-04-25T20:59:59.514957+0000","last_clean":"2024-04-25T20:59:59.514957+0000","last_became_active":"2024-04-25T20:56:51.276285+0000","last_became_peered":"2024-04-25T20:56:51.276285+0000","last_unstale":"2024-04-25T20:59:59.514957+0000","last_undegraded":"2024-04-25T20:59:59.514957+0000","last_fullsized":"2024-04-25T20:59:59.514957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:14.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119578+0000","last_change":"2024-04-25T20:56:48.946808+0000","last_active":"2024-04-25T20:59:59.119578+0000","last_peered":"2024-04-25T20:59:59.119578+0000","last_clean":"2024-04-25T20:59:59.119578+0000","last_became_active":"2024-04-25T20:56:48.946421+0000","last_became_peered":"2024-04-25T20:56:48.946421+0000","last_unstale":"2024-04-25T20:59:59.119578+0000","last_undegraded":"2024-04-25T20:59:59.119578+0000","last_fullsized":"2024-04-25T20:59:59.119578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:47.763950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266116+0000","last_change":"2024-04-25T20:56:51.274761+0000","last_active":"2024-04-25T20:59:59.266116+0000","last_peered":"2024-04-25T20:59:59.266116+0000","last_clean":"2024-04-25T20:59:59.266116+0000","last_became_active":"2024-04-25T20:56:51.274452+0000","last_became_peered":"2024-04-25T20:56:51.274452+0000","last_unstale":"2024-04-25T20:59:59.266116+0000","last_undegraded":"2024-04-25T20:59:59.266116+0000","last_fullsized":"2024-04-25T20:59:59.266116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:32.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":437,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:05.018237+0000","last_change":"2024-04-25T20:56:48.258982+0000","last_active":"2024-04-25T21:00:05.018237+0000","last_peered":"2024-04-25T21:00:05.018237+0000","last_clean":"2024-04-25T21:00:05.018237+0000","last_became_active":"2024-04-25T20:56:48.258593+0000","last_became_peered":"2024-04-25T20:56:48.258593+0000","last_unstale":"2024-04-25T21:00:05.018237+0000","last_undegraded":"2024-04-25T21:00:05.018237+0000","last_fullsized":"2024-04-25T21:00:05.018237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:20.518202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69970,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":370,"num_read_kb":254,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016324+0000","last_change":"2024-04-25T20:56:51.276361+0000","last_active":"2024-04-25T20:59:59.016324+0000","last_peered":"2024-04-25T20:59:59.016324+0000","last_clean":"2024-04-25T20:59:59.016324+0000","last_became_active":"2024-04-25T20:56:51.276045+0000","last_became_peered":"2024-04-25T20:56:51.276045+0000","last_unstale":"2024-04-25T20:59:59.016324+0000","last_undegraded":"2024-04-25T20:59:59.016324+0000","last_fullsized":"2024-04-25T20:59:59.016324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:04.325068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266011+0000","last_change":"2024-04-25T20:56:48.252358+0000","last_active":"2024-04-25T20:59:59.266011+0000","last_peered":"2024-04-25T20:59:59.266011+0000","last_clean":"2024-04-25T20:59:59.266011+0000","last_became_active":"2024-04-25T20:56:48.252027+0000","last_became_peered":"2024-04-25T20:56:48.252027+0000","last_unstale":"2024-04-25T20:59:59.266011+0000","last_undegraded":"2024-04-25T20:59:59.266011+0000","last_fullsized":"2024-04-25T20:59:59.266011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:03.024487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038654+0000","last_change":"2024-04-25T20:56:51.279599+0000","last_active":"2024-04-25T20:59:59.038654+0000","last_peered":"2024-04-25T20:59:59.038654+0000","last_clean":"2024-04-25T20:59:59.038654+0000","last_became_active":"2024-04-25T20:56:51.279294+0000","last_became_peered":"2024-04-25T20:56:51.279294+0000","last_unstale":"2024-04-25T20:59:59.038654+0000","last_undegraded":"2024-04-25T20:59:59.038654+0000","last_fullsized":"2024-04-25T20:59:59.038654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119216+0000","last_change":"2024-04-25T20:56:48.261005+0000","last_active":"2024-04-25T20:59:59.119216+0000","last_peered":"2024-04-25T20:59:59.119216+0000","last_clean":"2024-04-25T20:59:59.119216+0000","last_became_active":"2024-04-25T20:56:48.260543+0000","last_became_peered":"2024-04-25T20:56:48.260543+0000","last_unstale":"2024-04-25T20:59:59.119216+0000","last_undegraded":"2024-04-25T20:59:59.119216+0000","last_fullsized":"2024-04-25T20:59:59.119216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:41:54.326036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549230+0000","last_change":"2024-04-25T20:56:51.271753+0000","last_active":"2024-04-25T20:59:59.549230+0000","last_peered":"2024-04-25T20:59:59.549230+0000","last_clean":"2024-04-25T20:59:59.549230+0000","last_became_active":"2024-04-25T20:56:51.271392+0000","last_became_peered":"2024-04-25T20:56:51.271392+0000","last_unstale":"2024-04-25T20:59:59.549230+0000","last_undegraded":"2024-04-25T20:59:59.549230+0000","last_fullsized":"2024-04-25T20:59:59.549230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:26.605647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":215,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:03.047987+0000","last_change":"2024-04-25T20:56:48.260729+0000","last_active":"2024-04-25T21:00:03.047987+0000","last_peered":"2024-04-25T21:00:03.047987+0000","last_clean":"2024-04-25T21:00:03.047987+0000","last_became_active":"2024-04-25T20:56:48.260377+0000","last_became_peered":"2024-04-25T20:56:48.260377+0000","last_unstale":"2024-04-25T21:00:03.047987+0000","last_undegraded":"2024-04-25T21:00:03.047987+0000","last_fullsized":"2024-04-25T21:00:03.047987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:21.582266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.548948+0000","last_change":"2024-04-25T20:56:51.275691+0000","last_active":"2024-04-25T20:59:59.548948+0000","last_peered":"2024-04-25T20:59:59.548948+0000","last_clean":"2024-04-25T20:59:59.548948+0000","last_became_active":"2024-04-25T20:56:51.275220+0000","last_became_peered":"2024-04-25T20:56:51.275220+0000","last_unstale":"2024-04-25T20:59:59.548948+0000","last_undegraded":"2024-04-25T20:59:59.548948+0000","last_fullsized":"2024-04-25T20:59:59.548948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:24.367413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015925+0000","last_change":"2024-04-25T20:56:48.946465+0000","last_active":"2024-04-25T20:59:59.015925+0000","last_peered":"2024-04-25T20:59:59.015925+0000","last_clean":"2024-04-25T20:59:59.015925+0000","last_became_active":"2024-04-25T20:56:48.946093+0000","last_became_peered":"2024-04-25T20:56:48.946093+0000","last_unstale":"2024-04-25T20:59:59.015925+0000","last_undegraded":"2024-04-25T20:59:59.015925+0000","last_fullsized":"2024-04-25T20:59:59.015925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:29.502073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119264+0000","last_change":"2024-04-25T20:56:51.278464+0000","last_active":"2024-04-25T20:59:59.119264+0000","last_peered":"2024-04-25T20:59:59.119264+0000","last_clean":"2024-04-25T20:59:59.119264+0000","last_became_active":"2024-04-25T20:56:51.278119+0000","last_became_peered":"2024-04-25T20:56:51.278119+0000","last_unstale":"2024-04-25T20:59:59.119264+0000","last_undegraded":"2024-04-25T20:59:59.119264+0000","last_fullsized":"2024-04-25T20:59:59.119264+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:57:37.224817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016353+0000","last_change":"2024-04-25T20:56:48.255123+0000","last_active":"2024-04-25T20:59:59.016353+0000","last_peered":"2024-04-25T20:59:59.016353+0000","last_clean":"2024-04-25T20:59:59.016353+0000","last_became_active":"2024-04-25T20:56:48.254680+0000","last_became_peered":"2024-04-25T20:56:48.254680+0000","last_unstale":"2024-04-25T20:59:59.016353+0000","last_undegraded":"2024-04-25T20:59:59.016353+0000","last_fullsized":"2024-04-25T20:59:59.016353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:24.089590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113399+0000","last_change":"2024-04-25T20:57:47.592950+0000","last_active":"2024-04-25T20:59:59.113399+0000","last_peered":"2024-04-25T20:59:59.113399+0000","last_clean":"2024-04-25T20:59:59.113399+0000","last_became_active":"2024-04-25T20:57:47.592486+0000","last_became_peered":"2024-04-25T20:57:47.592486+0000","last_unstale":"2024-04-25T20:59:59.113399+0000","last_undegraded":"2024-04-25T20:59:59.113399+0000","last_fullsized":"2024-04-25T20:59:59.113399+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:43.218061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012933+0000","last_change":"2024-04-25T20:56:48.947508+0000","last_active":"2024-04-25T20:59:59.012933+0000","last_peered":"2024-04-25T20:59:59.012933+0000","last_clean":"2024-04-25T20:59:59.012933+0000","last_became_active":"2024-04-25T20:56:48.946582+0000","last_became_peered":"2024-04-25T20:56:48.946582+0000","last_unstale":"2024-04-25T20:59:59.012933+0000","last_undegraded":"2024-04-25T20:59:59.012933+0000","last_fullsized":"2024-04-25T20:59:59.012933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:11.198667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016523+0000","last_change":"2024-04-25T20:56:51.273560+0000","last_active":"2024-04-25T20:59:59.016523+0000","last_peered":"2024-04-25T20:59:59.016523+0000","last_clean":"2024-04-25T20:59:59.016523+0000","last_became_active":"2024-04-25T20:56:51.273182+0000","last_became_peered":"2024-04-25T20:56:51.273182+0000","last_unstale":"2024-04-25T20:59:59.016523+0000","last_undegraded":"2024-04-25T20:59:59.016523+0000","last_fullsized":"2024-04-25T20:59:59.016523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:47.012771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549107+0000","last_change":"2024-04-25T20:56:48.260754+0000","last_active":"2024-04-25T20:59:59.549107+0000","last_peered":"2024-04-25T20:59:59.549107+0000","last_clean":"2024-04-25T20:59:59.549107+0000","last_became_active":"2024-04-25T20:56:48.260420+0000","last_became_peered":"2024-04-25T20:56:48.260420+0000","last_unstale":"2024-04-25T20:59:59.549107+0000","last_undegraded":"2024-04-25T20:59:59.549107+0000","last_fullsized":"2024-04-25T20:59:59.549107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:00.219023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016420+0000","last_change":"2024-04-25T20:56:51.271944+0000","last_active":"2024-04-25T20:59:59.016420+0000","last_peered":"2024-04-25T20:59:59.016420+0000","last_clean":"2024-04-25T20:59:59.016420+0000","last_became_active":"2024-04-25T20:56:51.270656+0000","last_became_peered":"2024-04-25T20:56:51.270656+0000","last_unstale":"2024-04-25T20:59:59.016420+0000","last_undegraded":"2024-04-25T20:59:59.016420+0000","last_fullsized":"2024-04-25T20:59:59.016420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:00.624888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.012713+0000","last_change":"2024-04-25T20:57:47.526172+0000","last_active":"2024-04-25T20:59:59.012713+0000","last_peered":"2024-04-25T20:59:59.012713+0000","last_clean":"2024-04-25T20:59:59.012713+0000","last_became_active":"2024-04-25T20:57:47.525820+0000","last_became_peered":"2024-04-25T20:57:47.525820+0000","last_unstale":"2024-04-25T20:59:59.012713+0000","last_undegraded":"2024-04-25T20:59:59.012713+0000","last_fullsized":"2024-04-25T20:59:59.012713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:56.458959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":211,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:03.047120+0000","last_change":"2024-04-25T20:56:48.946471+0000","last_active":"2024-04-25T21:00:03.047120+0000","last_peered":"2024-04-25T21:00:03.047120+0000","last_clean":"2024-04-25T21:00:03.047120+0000","last_became_active":"2024-04-25T20:56:48.946029+0000","last_became_peered":"2024-04-25T20:56:48.946029+0000","last_unstale":"2024-04-25T21:00:03.047120+0000","last_undegraded":"2024-04-25T21:00:03.047120+0000","last_fullsized":"2024-04-25T21:00:03.047120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:30:16.400157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.013018+0000","last_change":"2024-04-25T20:57:48.300758+0000","last_active":"2024-04-25T20:59:59.013018+0000","last_peered":"2024-04-25T20:59:59.013018+0000","last_clean":"2024-04-25T20:59:59.013018+0000","last_became_active":"2024-04-25T20:57:48.300328+0000","last_became_peered":"2024-04-25T20:57:48.300328+0000","last_unstale":"2024-04-25T20:59:59.013018+0000","last_undegraded":"2024-04-25T20:59:59.013018+0000","last_fullsized":"2024-04-25T20:59:59.013018+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:31.931834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041515+0000","last_change":"2024-04-25T20:56:48.258479+0000","last_active":"2024-04-25T20:59:59.041515+0000","last_peered":"2024-04-25T20:59:59.041515+0000","last_clean":"2024-04-25T20:59:59.041515+0000","last_became_active":"2024-04-25T20:56:48.257951+0000","last_became_peered":"2024-04-25T20:56:48.257951+0000","last_unstale":"2024-04-25T20:59:59.041515+0000","last_undegraded":"2024-04-25T20:59:59.041515+0000","last_fullsized":"2024-04-25T20:59:59.041515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:13.812201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":403,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:01.134048+0000","last_change":"2024-04-25T20:55:28.701357+0000","last_active":"2024-04-25T21:00:01.134048+0000","last_peered":"2024-04-25T21:00:01.134048+0000","last_clean":"2024-04-25T21:00:01.134048+0000","last_became_active":"2024-04-25T20:55:28.700906+0000","last_became_peered":"2024-04-25T20:55:28.700906+0000","last_unstale":"2024-04-25T21:00:01.134048+0000","last_undegraded":"2024-04-25T21:00:01.134048+0000","last_fullsized":"2024-04-25T21:00:01.134048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:52:10.334323+0000","last_clean_scrub_stamp":"2024-04-25T20:52:10.334323+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:37:35.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038561+0000","last_change":"2024-04-25T20:56:51.275249+0000","last_active":"2024-04-25T20:59:59.038561+0000","last_peered":"2024-04-25T20:59:59.038561+0000","last_clean":"2024-04-25T20:59:59.038561+0000","last_became_active":"2024-04-25T20:56:51.274938+0000","last_became_peered":"2024-04-25T20:56:51.274938+0000","last_unstale":"2024-04-25T20:59:59.038561+0000","last_undegraded":"2024-04-25T20:59:59.038561+0000","last_fullsized":"2024-04-25T20:59:59.038561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:53.933324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549168+0000","last_change":"2024-04-25T20:56:48.259357+0000","last_active":"2024-04-25T20:59:59.549168+0000","last_peered":"2024-04-25T20:59:59.549168+0000","last_clean":"2024-04-25T20:59:59.549168+0000","last_became_active":"2024-04-25T20:56:48.258840+0000","last_became_peered":"2024-04-25T20:56:48.258840+0000","last_unstale":"2024-04-25T20:59:59.549168+0000","last_undegraded":"2024-04-25T20:59:59.549168+0000","last_fullsized":"2024-04-25T20:59:59.549168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:30.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016354+0000","last_change":"2024-04-25T20:56:48.946498+0000","last_active":"2024-04-25T20:59:59.016354+0000","last_peered":"2024-04-25T20:59:59.016354+0000","last_clean":"2024-04-25T20:59:59.016354+0000","last_became_active":"2024-04-25T20:56:48.946124+0000","last_became_peered":"2024-04-25T20:56:48.946124+0000","last_unstale":"2024-04-25T20:59:59.016354+0000","last_undegraded":"2024-04-25T20:59:59.016354+0000","last_fullsized":"2024-04-25T20:59:59.016354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:08.545574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038986+0000","last_change":"2024-04-25T20:56:51.276093+0000","last_active":"2024-04-25T20:59:59.038986+0000","last_peered":"2024-04-25T20:59:59.038986+0000","last_clean":"2024-04-25T20:59:59.038986+0000","last_became_active":"2024-04-25T20:56:51.275771+0000","last_became_peered":"2024-04-25T20:56:51.275771+0000","last_unstale":"2024-04-25T20:59:59.038986+0000","last_undegraded":"2024-04-25T20:59:59.038986+0000","last_fullsized":"2024-04-25T20:59:59.038986+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:15.159208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'103","reported_seq":236,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:01.628373+0000","last_change":"2024-04-25T20:57:47.533338+0000","last_active":"2024-04-25T21:00:01.628373+0000","last_peered":"2024-04-25T21:00:01.628373+0000","last_clean":"2024-04-25T21:00:01.628373+0000","last_became_active":"2024-04-25T20:57:47.529539+0000","last_became_peered":"2024-04-25T20:57:47.529539+0000","last_unstale":"2024-04-25T21:00:01.628373+0000","last_undegraded":"2024-04-25T21:00:01.628373+0000","last_fullsized":"2024-04-25T21:00:01.628373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:25.099128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":189,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042039+0000","last_change":"2024-04-25T20:56:51.275461+0000","last_active":"2024-04-25T20:59:59.042039+0000","last_peered":"2024-04-25T20:59:59.042039+0000","last_clean":"2024-04-25T20:59:59.042039+0000","last_became_active":"2024-04-25T20:56:51.275087+0000","last_became_peered":"2024-04-25T20:56:51.275087+0000","last_unstale":"2024-04-25T20:59:59.042039+0000","last_undegraded":"2024-04-25T20:59:59.042039+0000","last_fullsized":"2024-04-25T20:59:59.042039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:01:02.514490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041635+0000","last_change":"2024-04-25T20:56:49.031177+0000","last_active":"2024-04-25T20:59:59.041635+0000","last_peered":"2024-04-25T20:59:59.041635+0000","last_clean":"2024-04-25T20:59:59.041635+0000","last_became_active":"2024-04-25T20:56:49.030697+0000","last_became_peered":"2024-04-25T20:56:49.030697+0000","last_unstale":"2024-04-25T20:59:59.041635+0000","last_undegraded":"2024-04-25T20:59:59.041635+0000","last_fullsized":"2024-04-25T20:59:59.041635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:13.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514801+0000","last_change":"2024-04-25T20:57:47.795434+0000","last_active":"2024-04-25T20:59:59.514801+0000","last_peered":"2024-04-25T20:59:59.514801+0000","last_clean":"2024-04-25T20:59:59.514801+0000","last_became_active":"2024-04-25T20:57:47.795036+0000","last_became_peered":"2024-04-25T20:57:47.795036+0000","last_unstale":"2024-04-25T20:59:59.514801+0000","last_undegraded":"2024-04-25T20:59:59.514801+0000","last_fullsized":"2024-04-25T20:59:59.514801+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:48:56.749328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265934+0000","last_change":"2024-04-25T20:56:48.946689+0000","last_active":"2024-04-25T20:59:59.265934+0000","last_peered":"2024-04-25T20:59:59.265934+0000","last_clean":"2024-04-25T20:59:59.265934+0000","last_became_active":"2024-04-25T20:56:48.946229+0000","last_became_peered":"2024-04-25T20:56:48.946229+0000","last_unstale":"2024-04-25T20:59:59.265934+0000","last_undegraded":"2024-04-25T20:59:59.265934+0000","last_fullsized":"2024-04-25T20:59:59.265934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:04.120197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.015968+0000","last_change":"2024-04-25T20:56:51.275657+0000","last_active":"2024-04-25T20:59:59.015968+0000","last_peered":"2024-04-25T20:59:59.015968+0000","last_clean":"2024-04-25T20:59:59.015968+0000","last_became_active":"2024-04-25T20:56:51.275237+0000","last_became_peered":"2024-04-25T20:56:51.275237+0000","last_unstale":"2024-04-25T20:59:59.015968+0000","last_undegraded":"2024-04-25T20:59:59.015968+0000","last_fullsized":"2024-04-25T20:59:59.015968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:39.778340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041385+0000","last_change":"2024-04-25T20:56:48.251882+0000","last_active":"2024-04-25T20:59:59.041385+0000","last_peered":"2024-04-25T20:59:59.041385+0000","last_clean":"2024-04-25T20:59:59.041385+0000","last_became_active":"2024-04-25T20:56:48.251571+0000","last_became_peered":"2024-04-25T20:56:48.251571+0000","last_unstale":"2024-04-25T20:59:59.041385+0000","last_undegraded":"2024-04-25T20:59:59.041385+0000","last_fullsized":"2024-04-25T20:59:59.041385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:51.769332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113156+0000","last_change":"2024-04-25T20:56:51.280648+0000","last_active":"2024-04-25T20:59:59.113156+0000","last_peered":"2024-04-25T20:59:59.113156+0000","last_clean":"2024-04-25T20:59:59.113156+0000","last_became_active":"2024-04-25T20:56:51.280232+0000","last_became_peered":"2024-04-25T20:56:51.280232+0000","last_unstale":"2024-04-25T20:59:59.113156+0000","last_undegraded":"2024-04-25T20:59:59.113156+0000","last_fullsized":"2024-04-25T20:59:59.113156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:12.375519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113274+0000","last_change":"2024-04-25T20:57:47.535216+0000","last_active":"2024-04-25T20:59:59.113274+0000","last_peered":"2024-04-25T20:59:59.113274+0000","last_clean":"2024-04-25T20:59:59.113274+0000","last_became_active":"2024-04-25T20:57:47.530348+0000","last_became_peered":"2024-04-25T20:57:47.530348+0000","last_unstale":"2024-04-25T20:59:59.113274+0000","last_undegraded":"2024-04-25T20:59:59.113274+0000","last_fullsized":"2024-04-25T20:59:59.113274+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:01.373541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.042321+0000","last_change":"2024-04-25T20:56:51.277166+0000","last_active":"2024-04-25T20:59:59.042321+0000","last_peered":"2024-04-25T20:59:59.042321+0000","last_clean":"2024-04-25T20:59:59.042321+0000","last_became_active":"2024-04-25T20:56:51.276840+0000","last_became_peered":"2024-04-25T20:56:51.276840+0000","last_unstale":"2024-04-25T20:59:59.042321+0000","last_undegraded":"2024-04-25T20:59:59.042321+0000","last_fullsized":"2024-04-25T20:59:59.042321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:52.756660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.016118+0000","last_change":"2024-04-25T20:56:48.263426+0000","last_active":"2024-04-25T20:59:59.016118+0000","last_peered":"2024-04-25T20:59:59.016118+0000","last_clean":"2024-04-25T20:59:59.016118+0000","last_became_active":"2024-04-25T20:56:48.262832+0000","last_became_peered":"2024-04-25T20:56:48.262832+0000","last_unstale":"2024-04-25T20:59:59.016118+0000","last_undegraded":"2024-04-25T20:59:59.016118+0000","last_fullsized":"2024-04-25T20:59:59.016118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:09.362161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549131+0000","last_change":"2024-04-25T20:56:51.274267+0000","last_active":"2024-04-25T20:59:59.549131+0000","last_peered":"2024-04-25T20:59:59.549131+0000","last_clean":"2024-04-25T20:59:59.549131+0000","last_became_active":"2024-04-25T20:56:51.273807+0000","last_became_peered":"2024-04-25T20:56:51.273807+0000","last_unstale":"2024-04-25T20:59:59.549131+0000","last_undegraded":"2024-04-25T20:59:59.549131+0000","last_fullsized":"2024-04-25T20:59:59.549131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266148+0000","last_change":"2024-04-25T20:56:49.031011+0000","last_active":"2024-04-25T20:59:59.266148+0000","last_peered":"2024-04-25T20:59:59.266148+0000","last_clean":"2024-04-25T20:59:59.266148+0000","last_became_active":"2024-04-25T20:56:49.030522+0000","last_became_peered":"2024-04-25T20:56:49.030522+0000","last_unstale":"2024-04-25T20:59:59.266148+0000","last_undegraded":"2024-04-25T20:59:59.266148+0000","last_fullsized":"2024-04-25T20:59:59.266148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:00.915454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549501+0000","last_change":"2024-04-25T20:56:51.276989+0000","last_active":"2024-04-25T20:59:59.549501+0000","last_peered":"2024-04-25T20:59:59.549501+0000","last_clean":"2024-04-25T20:59:59.549501+0000","last_became_active":"2024-04-25T20:56:51.275417+0000","last_became_peered":"2024-04-25T20:56:51.275417+0000","last_unstale":"2024-04-25T20:59:59.549501+0000","last_undegraded":"2024-04-25T20:59:59.549501+0000","last_fullsized":"2024-04-25T20:59:59.549501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:27.516062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'65","reported_seq":124,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T21:00:01.247407+0000","last_change":"2024-04-25T20:56:48.253713+0000","last_active":"2024-04-25T21:00:01.247407+0000","last_peered":"2024-04-25T21:00:01.247407+0000","last_clean":"2024-04-25T21:00:01.247407+0000","last_became_active":"2024-04-25T20:56:48.253215+0000","last_became_peered":"2024-04-25T20:56:48.253215+0000","last_unstale":"2024-04-25T21:00:01.247407+0000","last_undegraded":"2024-04-25T21:00:01.247407+0000","last_fullsized":"2024-04-25T21:00:01.247407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:57.946840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549273+0000","last_change":"2024-04-25T20:56:51.274691+0000","last_active":"2024-04-25T20:59:59.549273+0000","last_peered":"2024-04-25T20:59:59.549273+0000","last_clean":"2024-04-25T20:59:59.549273+0000","last_became_active":"2024-04-25T20:56:51.274350+0000","last_became_peered":"2024-04-25T20:56:51.274350+0000","last_unstale":"2024-04-25T20:59:59.549273+0000","last_undegraded":"2024-04-25T20:59:59.549273+0000","last_fullsized":"2024-04-25T20:59:59.549273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:52.375409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039075+0000","last_change":"2024-04-25T20:56:48.251616+0000","last_active":"2024-04-25T20:59:59.039075+0000","last_peered":"2024-04-25T20:59:59.039075+0000","last_clean":"2024-04-25T20:59:59.039075+0000","last_became_active":"2024-04-25T20:56:48.251248+0000","last_became_peered":"2024-04-25T20:56:48.251248+0000","last_unstale":"2024-04-25T20:59:59.039075+0000","last_undegraded":"2024-04-25T20:59:59.039075+0000","last_fullsized":"2024-04-25T20:59:59.039075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:47.231199+0000","last_clean_scrub_stamp":"2024-04-25T20:56:47.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:51.032356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.549388+0000","last_change":"2024-04-25T20:56:51.278372+0000","last_active":"2024-04-25T20:59:59.549388+0000","last_peered":"2024-04-25T20:59:59.549388+0000","last_clean":"2024-04-25T20:59:59.549388+0000","last_became_active":"2024-04-25T20:56:51.277989+0000","last_became_peered":"2024-04-25T20:56:51.277989+0000","last_unstale":"2024-04-25T20:59:59.549388+0000","last_undegraded":"2024-04-25T20:59:59.549388+0000","last_fullsized":"2024-04-25T20:59:59.549388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.017713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113230+0000","last_change":"2024-04-25T20:56:51.276270+0000","last_active":"2024-04-25T20:59:59.113230+0000","last_peered":"2024-04-25T20:59:59.113230+0000","last_clean":"2024-04-25T20:59:59.113230+0000","last_became_active":"2024-04-25T20:56:51.275948+0000","last_became_peered":"2024-04-25T20:56:51.275948+0000","last_unstale":"2024-04-25T20:59:59.113230+0000","last_undegraded":"2024-04-25T20:59:59.113230+0000","last_fullsized":"2024-04-25T20:59:59.113230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:33.548439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265845+0000","last_change":"2024-04-25T20:56:51.283931+0000","last_active":"2024-04-25T20:59:59.265845+0000","last_peered":"2024-04-25T20:59:59.265845+0000","last_clean":"2024-04-25T20:59:59.265845+0000","last_became_active":"2024-04-25T20:56:51.283120+0000","last_became_peered":"2024-04-25T20:56:51.283120+0000","last_unstale":"2024-04-25T20:59:59.265845+0000","last_undegraded":"2024-04-25T20:59:59.265845+0000","last_fullsized":"2024-04-25T20:59:59.265845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:26.676750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.265827+0000","last_change":"2024-04-25T20:56:51.286013+0000","last_active":"2024-04-25T20:59:59.265827+0000","last_peered":"2024-04-25T20:59:59.265827+0000","last_clean":"2024-04-25T20:59:59.265827+0000","last_became_active":"2024-04-25T20:56:51.285645+0000","last_became_peered":"2024-04-25T20:56:51.285645+0000","last_unstale":"2024-04-25T20:59:59.265827+0000","last_undegraded":"2024-04-25T20:59:59.265827+0000","last_fullsized":"2024-04-25T20:59:59.265827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:51.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.514609+0000","last_change":"2024-04-25T20:56:51.272567+0000","last_active":"2024-04-25T20:59:59.514609+0000","last_peered":"2024-04-25T20:59:59.514609+0000","last_clean":"2024-04-25T20:59:59.514609+0000","last_became_active":"2024-04-25T20:56:51.271427+0000","last_became_peered":"2024-04-25T20:56:51.271427+0000","last_unstale":"2024-04-25T20:59:59.514609+0000","last_undegraded":"2024-04-25T20:59:59.514609+0000","last_fullsized":"2024-04-25T20:59:59.514609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:19.945982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.038896+0000","last_change":"2024-04-25T20:56:51.272448+0000","last_active":"2024-04-25T20:59:59.038896+0000","last_peered":"2024-04-25T20:59:59.038896+0000","last_clean":"2024-04-25T20:59:59.038896+0000","last_became_active":"2024-04-25T20:56:51.271965+0000","last_became_peered":"2024-04-25T20:56:51.271965+0000","last_unstale":"2024-04-25T20:59:59.038896+0000","last_undegraded":"2024-04-25T20:59:59.038896+0000","last_fullsized":"2024-04-25T20:59:59.038896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:03.766053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113416+0000","last_change":"2024-04-25T20:56:51.280260+0000","last_active":"2024-04-25T20:59:59.113416+0000","last_peered":"2024-04-25T20:59:59.113416+0000","last_clean":"2024-04-25T20:59:59.113416+0000","last_became_active":"2024-04-25T20:56:51.279864+0000","last_became_peered":"2024-04-25T20:56:51.279864+0000","last_unstale":"2024-04-25T20:59:59.113416+0000","last_undegraded":"2024-04-25T20:59:59.113416+0000","last_fullsized":"2024-04-25T20:59:59.113416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.136824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266115+0000","last_change":"2024-04-25T20:56:51.279041+0000","last_active":"2024-04-25T20:59:59.266115+0000","last_peered":"2024-04-25T20:59:59.266115+0000","last_clean":"2024-04-25T20:59:59.266115+0000","last_became_active":"2024-04-25T20:56:51.278302+0000","last_became_peered":"2024-04-25T20:56:51.278302+0000","last_unstale":"2024-04-25T20:59:59.266115+0000","last_undegraded":"2024-04-25T20:59:59.266115+0000","last_fullsized":"2024-04-25T20:59:59.266115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:02.510919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041907+0000","last_change":"2024-04-25T20:56:51.275618+0000","last_active":"2024-04-25T20:59:59.041907+0000","last_peered":"2024-04-25T20:59:59.041907+0000","last_clean":"2024-04-25T20:59:59.041907+0000","last_became_active":"2024-04-25T20:56:51.275249+0000","last_became_peered":"2024-04-25T20:56:51.275249+0000","last_unstale":"2024-04-25T20:59:59.041907+0000","last_undegraded":"2024-04-25T20:59:59.041907+0000","last_fullsized":"2024-04-25T20:59:59.041907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:01.793083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.119251+0000","last_change":"2024-04-25T20:56:51.275868+0000","last_active":"2024-04-25T20:59:59.119251+0000","last_peered":"2024-04-25T20:59:59.119251+0000","last_clean":"2024-04-25T20:59:59.119251+0000","last_became_active":"2024-04-25T20:56:51.275544+0000","last_became_peered":"2024-04-25T20:56:51.275544+0000","last_unstale":"2024-04-25T20:59:59.119251+0000","last_undegraded":"2024-04-25T20:59:59.119251+0000","last_fullsized":"2024-04-25T20:59:59.119251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:31.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.041380+0000","last_change":"2024-04-25T20:56:51.274963+0000","last_active":"2024-04-25T20:59:59.041380+0000","last_peered":"2024-04-25T20:59:59.041380+0000","last_clean":"2024-04-25T20:59:59.041380+0000","last_became_active":"2024-04-25T20:56:51.274597+0000","last_became_peered":"2024-04-25T20:56:51.274597+0000","last_unstale":"2024-04-25T20:59:59.041380+0000","last_undegraded":"2024-04-25T20:59:59.041380+0000","last_fullsized":"2024-04-25T20:59:59.041380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:14.464560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.266007+0000","last_change":"2024-04-25T20:56:51.279602+0000","last_active":"2024-04-25T20:59:59.266007+0000","last_peered":"2024-04-25T20:59:59.266007+0000","last_clean":"2024-04-25T20:59:59.266007+0000","last_became_active":"2024-04-25T20:56:51.279221+0000","last_became_peered":"2024-04-25T20:56:51.279221+0000","last_unstale":"2024-04-25T20:59:59.266007+0000","last_undegraded":"2024-04-25T20:59:59.266007+0000","last_fullsized":"2024-04-25T20:59:59.266007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:26.960492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.039002+0000","last_change":"2024-04-25T20:56:51.280465+0000","last_active":"2024-04-25T20:59:59.039002+0000","last_peered":"2024-04-25T20:59:59.039002+0000","last_clean":"2024-04-25T20:59:59.039002+0000","last_became_active":"2024-04-25T20:56:51.280168+0000","last_became_peered":"2024-04-25T20:56:51.280168+0000","last_unstale":"2024-04-25T20:59:59.039002+0000","last_undegraded":"2024-04-25T20:59:59.039002+0000","last_fullsized":"2024-04-25T20:59:59.039002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:04.211113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:59:59.113237+0000","last_change":"2024-04-25T20:56:51.283084+0000","last_active":"2024-04-25T20:59:59.113237+0000","last_peered":"2024-04-25T20:59:59.113237+0000","last_clean":"2024-04-25T20:59:59.113237+0000","last_became_active":"2024-04-25T20:56:51.282692+0000","last_became_peered":"2024-04-25T20:56:51.282692+0000","last_unstale":"2024-04-25T20:59:59.113237+0000","last_undegraded":"2024-04-25T20:59:59.113237+0000","last_fullsized":"2024-04-25T20:59:59.113237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:56:50.248053+0000","last_clean_scrub_stamp":"2024-04-25T20:56:50.248053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:13:27.527112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796400,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1444,"num_read_kb":939,"num_write":423,"num_write_kb":2357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2389200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1409024,"data_stored":886419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37460,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707692,"statfs":{"total":95995035648,"available":95956676608,"internally_reserved":0,"allocated":2052096,"data_stored":1483726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776192,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1425408,"data_stored":886655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037829,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":493359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1626112,"data_stored":1086270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1019904,"data_stored":492828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":562591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":494099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":564473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28856,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716296,"statfs":{"total":95995035648,"available":95965487104,"internally_reserved":0,"allocated":1761280,"data_stored":1214587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920856,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":494105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724628,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33220,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711932,"statfs":{"total":95995035648,"available":95961018368,"internally_reserved":0,"allocated":1970176,"data_stored":1413346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":400620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:00:10.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:00:10.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:10 smithi186 ceph-mon[32529]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-25T21:00:10.989 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:11.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:10 smithi092 ceph-mon[32524]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-25T21:00:11.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:11 smithi186 ceph-mon[32529]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:12.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:11 smithi092 ceph-mon[32524]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:12.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:11 smithi047 ceph-mon[29195]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 -- 172.21.15.47:0/962096646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 msgr2=0x7fc97c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 --2- 172.21.15.47:0/962096646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 0x7fc97c108e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009990 tx=0x7fc96c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 -- 172.21.15.47:0/962096646 shutdown_connections 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 --2- 172.21.15.47:0/962096646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 0x7fc97c108e50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 --2- 172.21.15.47:0/962096646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c103480 0x7fc97c10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 --2- 172.21.15.47:0/962096646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc97c102ac0 0x7fc97c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 -- 172.21.15.47:0/962096646 >> 172.21.15.47:0/962096646 conn(0x7fc97c079bb0 msgr2=0x7fc97c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 -- 172.21.15.47:0/962096646 shutdown_connections 2024-04-25T21:00:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 -- 172.21.15.47:0/962096646 wait complete. 2024-04-25T21:00:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.623+0000 7fc982638700 1 Processor -- start 2024-04-25T21:00:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc982638700 1 -- start start 2024-04-25T21:00:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc982638700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c102ac0 0x7fc97c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc982638700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc97c103480 0x7fc97c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc982638700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 0x7fc97c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc982638700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c112c20 con 0x7fc97c108a70 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc982638700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c112aa0 con 0x7fc97c102ac0 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc982638700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c112da0 con 0x7fc97c103480 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc980bd5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 0x7fc97c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc980bd5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 0x7fc97c19fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42304/0 (socket says 172.21.15.47:42304) 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc97bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c102ac0 0x7fc97c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc980bd5700 1 -- 172.21.15.47:0/1443925449 learned_addr learned my addr 172.21.15.47:0/1443925449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc97b7fe700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc97c103480 0x7fc97c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc980bd5700 1 -- 172.21.15.47:0/1443925449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc97c103480 msgr2=0x7fc97c19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc980bd5700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc97c103480 0x7fc97c19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc980bd5700 1 -- 172.21.15.47:0/1443925449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c102ac0 msgr2=0x7fc97c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc980bd5700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c102ac0 0x7fc97c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.624+0000 7fc980bd5700 1 -- 172.21.15.47:0/1443925449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c1a02d0 con 0x7fc97c108a70 2024-04-25T21:00:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.625+0000 7fc980bd5700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 0x7fc97c19fb90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc97400ef30 tx=0x7fc97400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.625+0000 7fc9797fa700 1 -- 172.21.15.47:0/1443925449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740092e0 con 0x7fc97c108a70 2024-04-25T21:00:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.625+0000 7fc9797fa700 1 -- 172.21.15.47:0/1443925449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc974004bb0 con 0x7fc97c108a70 2024-04-25T21:00:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.625+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c1a0560 con 0x7fc97c108a70 2024-04-25T21:00:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.625+0000 7fc9797fa700 1 -- 172.21.15.47:0/1443925449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974004020 con 0x7fc97c108a70 2024-04-25T21:00:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.625+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c1a7e40 con 0x7fc97c108a70 2024-04-25T21:00:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.626+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c1041a0 con 0x7fc97c108a70 2024-04-25T21:00:12.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.627+0000 7fc9797fa700 1 -- 172.21.15.47:0/1443925449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7fc97401b030 con 0x7fc97c108a70 2024-04-25T21:00:12.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.628+0000 7fc9797fa700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc9640a3ed0 0x7fc9640a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.628+0000 7fc97bfff700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc9640a3ed0 0x7fc9640a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.628+0000 7fc9797fa700 1 -- 172.21.15.47:0/1443925449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fc9740d0e80 con 0x7fc97c108a70 2024-04-25T21:00:12.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.628+0000 7fc97bfff700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc9640a3ed0 0x7fc9640a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098e0 tx=0x7fc96c005960 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.632+0000 7fc9797fa700 1 -- 172.21.15.47:0/1443925449 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc974022020 con 0x7fc97c108a70 2024-04-25T21:00:12.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.889+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc97c19c620 con 0x7fc97c108a70 2024-04-25T21:00:12.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.890+0000 7fc9797fa700 1 -- 172.21.15.47:0/1443925449 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11853 (secure 0 0 0) 0x7fc9740a3070 con 0x7fc97c108a70 2024-04-25T21:00:12.890 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:00:12.891 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:59:57.345381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc9640a3ed0 msgr2=0x7fc9640a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc9640a3ed0 0x7fc9640a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098e0 tx=0x7fc96c005960 comp rx=0 tx=0).stop 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 msgr2=0x7fc97c19fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 0x7fc97c19fb90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc97400ef30 tx=0x7fc97400c3a0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 shutdown_connections 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc9640a3ed0 0x7fc9640a6390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97c108a70 0x7fc97c19fb90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c102ac0 0x7fc97c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 --2- 172.21.15.47:0/1443925449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc97c103480 0x7fc97c19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 >> 172.21.15.47:0/1443925449 conn(0x7fc97c079bb0 msgr2=0x7fc97c10df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 shutdown_connections 2024-04-25T21:00:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:12.892+0000 7fc982638700 1 -- 172.21.15.47:0/1443925449 wait complete. 2024-04-25T21:00:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-04-25T21:00:12.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:12 smithi186 ceph-mon[32529]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T21:00:13.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:12 smithi092 ceph-mon[32524]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T21:00:13.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:12 smithi047 ceph-mon[29195]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T21:00:13.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:00:13.739 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:13.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:13 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1443925449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:13.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:13 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1443925449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:13 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1443925449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 -- 172.21.15.47:0/1258139159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 msgr2=0x7f26d410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 --2- 172.21.15.47:0/1258139159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 0x7f26d410ee60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009ca0 tx=0x7f26cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 -- 172.21.15.47:0/1258139159 shutdown_connections 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 --2- 172.21.15.47:0/1258139159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 0x7f26d410ee60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 --2- 172.21.15.47:0/1258139159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d4102b70 0x7f26d4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 --2- 172.21.15.47:0/1258139159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26d41021c0 0x7f26d41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 -- 172.21.15.47:0/1258139159 >> 172.21.15.47:0/1258139159 conn(0x7f26d40fda20 msgr2=0x7f26d40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 -- 172.21.15.47:0/1258139159 shutdown_connections 2024-04-25T21:00:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.913+0000 7f26d92a9640 1 -- 172.21.15.47:0/1258139159 wait complete. 2024-04-25T21:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d92a9640 1 Processor -- start 2024-04-25T21:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d92a9640 1 -- start start 2024-04-25T21:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d92a9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d41021c0 0x7f26d4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d92a9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 0x7f26d4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d92a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26d419cf20 0x7f26d419d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d92a9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41121e0 con 0x7f26d4107290 2024-04-25T21:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d92a9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4112060 con 0x7f26d419cf20 2024-04-25T21:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d92a9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4112360 con 0x7f26d41021c0 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 0x7f26d4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 0x7f26d4198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51418/0 (socket says 172.21.15.47:51418) 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.914+0000 7f26d27fc640 1 -- 172.21.15.47:0/147642091 learned_addr learned my addr 172.21.15.47:0/147642091 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d37fe640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26d419cf20 0x7f26d419d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d41021c0 0x7f26d4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d27fc640 1 -- 172.21.15.47:0/147642091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d41021c0 msgr2=0x7f26d4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d27fc640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d41021c0 0x7f26d4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d27fc640 1 -- 172.21.15.47:0/147642091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26d419cf20 msgr2=0x7f26d419d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d27fc640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26d419cf20 0x7f26d419d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d27fc640 1 -- 172.21.15.47:0/147642091 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d419da10 con 0x7f26d4107290 2024-04-25T21:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d37fe640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26d419cf20 0x7f26d419d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d27fc640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 0x7f26d4198ba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f26c800d770 tx=0x7f26c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26bbfff640 1 -- 172.21.15.47:0/147642091 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8017070 con 0x7f26d4107290 2024-04-25T21:00:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d92a9640 1 -- 172.21.15.47:0/147642091 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d419dd00 con 0x7f26d4107290 2024-04-25T21:00:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26bbfff640 1 -- 172.21.15.47:0/147642091 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c8009c30 con 0x7f26d4107290 2024-04-25T21:00:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26bbfff640 1 -- 172.21.15.47:0/147642091 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c800f620 con 0x7f26d4107290 2024-04-25T21:00:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.915+0000 7f26d92a9640 1 -- 172.21.15.47:0/147642091 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d41a54f0 con 0x7f26d4107290 2024-04-25T21:00:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.917+0000 7f26bbfff640 1 -- 172.21.15.47:0/147642091 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f26c801a070 con 0x7f26d4107290 2024-04-25T21:00:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26dbfbd740 1 Processor -- start 2024-04-25T21:00:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26dbfbd740 1 -- start start 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 0x565060d2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2df40 0x565060d382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d3abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x565060c92970 con 0x565060d38800 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565060c2f4a0 con 0x565060d2df40 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x565060c8e930 con 0x565060d2d620 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26bbfff640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f26bc0a3ee0 0x7f26bc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d3abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d3abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51432/0 (socket says 172.21.15.47:51432) 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d27fc640 1 -- 172.21.15.47:0/1455074286 learned_addr learned my addr 172.21.15.47:0/1455074286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d37fe640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f26bc0a3ee0 0x7f26bc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d37fe640 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 0x565060d2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2df40 0x565060d382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26bbfff640 1 -- 172.21.15.47:0/147642091 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f26c80d0630 con 0x7f26d4107290 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d27fc640 1 -- 172.21.15.47:0/1455074286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 msgr2=0x565060d2da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d27fc640 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 0x565060d2da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d27fc640 1 -- 172.21.15.47:0/1455074286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2df40 msgr2=0x565060d382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.918+0000 7f26d27fc640 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2df40 0x565060d382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.919+0000 7f26d27fc640 1 -- 172.21.15.47:0/1455074286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650609a5210 con 0x565060d38800 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.919+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2df40 0x565060d382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.919+0000 7f26d37fe640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f26bc0a3ee0 0x7f26bc0a63a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009a60 tx=0x7f26cc005c50 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.919+0000 7f26d27fc640 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d3abf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f26c8098d60 tx=0x7f26c800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0a96046fd562e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.919+0000 7f26b97fa640 1 -- 172.21.15.47:0/1455074286 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8021070 con 0x565060d38800 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.919+0000 7f26b97fa640 1 -- 172.21.15.47:0/1455074286 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c8009de0 con 0x565060d38800 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.919+0000 7f26b97fa640 1 -- 172.21.15.47:0/1455074286 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8098ba0 con 0x565060d38800 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- 172.21.15.47:0/1455074286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 msgr2=0x565060d3abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d3abf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f26c8098d60 tx=0x7f26c800de80 comp rx=0 tx=0).stop 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- 172.21.15.47:0/1455074286 shutdown_connections 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d3abf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2df40 0x565060d382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/1455074286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 0x565060d2da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- 172.21.15.47:0/1455074286 >> 172.21.15.47:0/1455074286 conn(0x565060d29b30 msgr2=0x565060d29f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- 172.21.15.47:0/1455074286 shutdown_connections 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- 172.21.15.47:0/1455074286 wait complete. 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- start start 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 Processor -- start 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 0x565060d38200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2df40 0x565060d354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 0x565060d35b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x565060d3b3d0 con 0x565060d2df40 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565060c2d0b0 con 0x565060d38800 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x565060d3b550 con 0x565060d2d620 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26dbfbd740 10 client.0 _ll_register_callbacks cb 0x565060c1ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2df40 0x565060d354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.920+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2df40 0x565060d354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51438/0 (socket says 172.21.15.47:51438) 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d27fc640 1 -- 172.21.15.47:0/241083849 learned_addr learned my addr 172.21.15.47:0/241083849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d37fe640 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 0x565060d35b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 0x565060d38200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d27fc640 1 -- 172.21.15.47:0/241083849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 msgr2=0x565060d38200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d27fc640 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 0x565060d38200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d27fc640 1 -- 172.21.15.47:0/241083849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 msgr2=0x565060d35b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d27fc640 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 0x565060d35b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d27fc640 1 -- 172.21.15.47:0/241083849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565060a71620 con 0x565060d2df40 2024-04-25T21:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26d27fc640 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2df40 0x565060d354f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26c80cdd20 tx=0x7f26c809c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39eecccef1d5df8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26b6ff5640 1 -- 172.21.15.47:0/241083849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8021050 con 0x565060d2df40 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26dbfbd740 1 -- 172.21.15.47:0/241083849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c8009de0 con 0x565060d2df40 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26b6ff5640 1 -- 172.21.15.47:0/241083849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f26c80a4d70 con 0x565060d2df40 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26b6ff5640 1 -- 172.21.15.47:0/241083849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26c80a0740 con 0x565060d2df40 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26dbfbd740 10 client.15270 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26dbfbd740 1 -- 172.21.15.47:0/241083849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565060d47380 con 0x565060d2df40 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.921+0000 7f26b6ff5640 1 -- 172.21.15.47:0/241083849 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c8009de0 con 0x565060d2df40 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26b6ff5640 1 -- 172.21.15.47:0/241083849 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f26c80d0ae0 con 0x565060d2df40 2024-04-25T21:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26dbfbd740 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26dbfbd740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26dbfbd740 20 client.15270 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x565060d486d0 0x565060d4aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26dbfbd740 4 client.15270 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26dbfbd740 1 -- 172.21.15.47:0/241083849 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565060ca0390 con 0x565060d486d0 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x565060d486d0 0x565060d4aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x565060d486d0 0x565060d4aac0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a9429f160702fe server_cookie=4f9644b4e699d38e in_seq=0 out_seq=0 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.922+0000 7f26b6ff5640 10 client.15270 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26b6ff5640 1 -- 172.21.15.47:0/241083849 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565060ca0390 con 0x565060d486d0 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26b6ff5640 10 client.15270 handle_command_reply: tid=0 2024-04-25T21:00:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 client.15270 shutdown 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 -- 172.21.15.47:0/241083849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2df40 msgr2=0x565060d354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2df40 0x565060d354f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26c80cdd20 tx=0x7f26c809c390 comp rx=0 tx=0).stop 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 -- 172.21.15.47:0/241083849 shutdown_connections 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x565060d486d0 0x565060d4aac0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 0x565060d35b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2df40 0x565060d354f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/241083849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2d620 0x565060d38200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 -- 172.21.15.47:0/241083849 >> 172.21.15.47:0/241083849 conn(0x565060d29b30 msgr2=0x565060d28cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26b6ff5640 0 client.15270 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 -- 172.21.15.47:0/241083849 shutdown_connections 2024-04-25T21:00:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 1 -- 172.21.15.47:0/241083849 wait complete. 2024-04-25T21:00:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.923+0000 7f26dbfbd740 20 client.15270 trim_cache size 0 max 16384 2024-04-25T21:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26dbfbd740 1 Processor -- start 2024-04-25T21:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26dbfbd740 1 -- start start 2024-04-25T21:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2d620 0x565060d4d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060d4dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d32750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41b0330 con 0x565060d38800 2024-04-25T21:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41adc00 con 0x565060d2d620 2024-04-25T21:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41aba70 con 0x565060d2df40 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26d37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d32750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26d37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d32750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51454/0 (socket says 172.21.15.47:51454) 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.938+0000 7f26d37fe640 1 -- 172.21.15.47:0/3530588580 learned_addr learned my addr 172.21.15.47:0/3530588580 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2d620 0x565060d4d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d37fe640 1 -- 172.21.15.47:0/3530588580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 msgr2=0x565060d4dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d27fc640 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060d4dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d37fe640 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060d4dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d37fe640 1 -- 172.21.15.47:0/3530588580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2d620 msgr2=0x565060d4d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d37fe640 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2d620 0x565060d4d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d37fe640 1 -- 172.21.15.47:0/3530588580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565060d344a0 con 0x565060d38800 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2d620 0x565060d4d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d27fc640 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060d4dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26d37fe640 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d32750 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02ade0 tx=0x7f26cc045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59e1f5829c2c4b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26b8ff9640 1 -- 172.21.15.47:0/3530588580 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc031e30 con 0x565060d38800 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26b8ff9640 1 -- 172.21.15.47:0/3530588580 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26cc036040 con 0x565060d38800 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.939+0000 7f26b8ff9640 1 -- 172.21.15.47:0/3530588580 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0047e0 con 0x565060d38800 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- 172.21.15.47:0/3530588580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 msgr2=0x565060d32750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d32750 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02ade0 tx=0x7f26cc045000 comp rx=0 tx=0).stop 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- 172.21.15.47:0/3530588580 shutdown_connections 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d38800 0x565060d32750 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060d4dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/3530588580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d2d620 0x565060d4d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- 172.21.15.47:0/3530588580 >> 172.21.15.47:0/3530588580 conn(0x565060d29b30 msgr2=0x565060d29f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- 172.21.15.47:0/3530588580 shutdown_connections 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- 172.21.15.47:0/3530588580 wait complete. 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- start start 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 Processor -- start 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2d620 0x565060dab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060dab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 0x565060da5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ac0c0 con 0x565060d2d620 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ad530 con 0x565060d38800 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ac790 con 0x565060d2df40 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26dbfbd740 10 client.0 _ll_register_callbacks cb 0x7f26c809bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2d620 0x565060dab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2d620 0x565060dab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51470/0 (socket says 172.21.15.47:51470) 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.940+0000 7f26d2ffd640 1 -- 172.21.15.47:0/2566081928 learned_addr learned my addr 172.21.15.47:0/2566081928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26d27fc640 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060dab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26d2ffd640 1 -- 172.21.15.47:0/2566081928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 msgr2=0x565060dab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060dab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26d2ffd640 1 -- 172.21.15.47:0/2566081928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 msgr2=0x565060da5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 0x565060da5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26d2ffd640 1 -- 172.21.15.47:0/2566081928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565060da5d50 con 0x565060d2d620 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26d27fc640 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060dab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26d2ffd640 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2d620 0x565060dab250 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x565060c2f5c0 tx=0x7f26c4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bec65956a629bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26b6ff5640 1 -- 172.21.15.47:0/2566081928 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c401e880 con 0x565060d2d620 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26b6ff5640 1 -- 172.21.15.47:0/2566081928 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f26c4011440 con 0x565060d2d620 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26dbfbd740 1 -- 172.21.15.47:0/2566081928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26cc036040 con 0x565060d2d620 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26b6ff5640 1 -- 172.21.15.47:0/2566081928 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26c402d050 con 0x565060d2d620 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26dbfbd740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26dbfbd740 1 -- 172.21.15.47:0/2566081928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565060db2a30 con 0x565060d2d620 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26b6ff5640 1 -- 172.21.15.47:0/2566081928 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26cc036040 con 0x565060d2d620 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.941+0000 7f26b6ff5640 1 -- 172.21.15.47:0/2566081928 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f26c400f6c0 con 0x565060d2d620 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26dbfbd740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26dbfbd740 10 client.15276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26dbfbd740 20 client.15276 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x565060db31b0 0x565060db5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26dbfbd740 4 client.15276 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:00:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26dbfbd740 1 -- 172.21.15.47:0/2566081928 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565060c9a200 con 0x565060db31b0 2024-04-25T21:00:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26d27fc640 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x565060db31b0 0x565060db5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26b6ff5640 10 client.15276 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.942+0000 7f26d27fc640 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x565060db31b0 0x565060db5610 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17d978b72dca204 server_cookie=a77f1af2758a9a99 in_seq=0 out_seq=0 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.955+0000 7f26b6ff5640 1 -- 172.21.15.47:0/2566081928 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565060c9a200 con 0x565060db31b0 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.955+0000 7f26b6ff5640 10 client.15276 handle_command_reply: tid=0 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.955+0000 7f26dbfbd740 1 client.15276 shutdown 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.955+0000 7f26dbfbd740 1 -- 172.21.15.47:0/2566081928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2d620 msgr2=0x565060dab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.955+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2d620 0x565060dab250 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x565060c2f5c0 tx=0x7f26c4012820 comp rx=0 tx=0).stop 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 1 -- 172.21.15.47:0/2566081928 shutdown_connections 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x565060db31b0 0x565060db5610 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565060d38800 0x565060da5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565060d2df40 0x565060dab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26b6ff5640 0 client.15276 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 1 --2- 172.21.15.47:0/2566081928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565060d2d620 0x565060dab250 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 1 -- 172.21.15.47:0/2566081928 >> 172.21.15.47:0/2566081928 conn(0x565060d29b30 msgr2=0x565060d28420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 1 -- 172.21.15.47:0/2566081928 shutdown_connections 2024-04-25T21:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 1 -- 172.21.15.47:0/2566081928 wait complete. 2024-04-25T21:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26dbfbd740 20 client.15276 trim_cache size 0 max 16384 2024-04-25T21:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:00:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:00:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 -- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f26bc0a3ee0 msgr2=0x7f26bc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f26bc0a3ee0 0x7f26bc0a63a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009a60 tx=0x7f26cc005c50 comp rx=0 tx=0).stop 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 -- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 msgr2=0x7f26d4198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 0x7f26d4198ba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f26c800d770 tx=0x7f26c800dc30 comp rx=0 tx=0).stop 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 -- 172.21.15.47:0/147642091 shutdown_connections 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f26bc0a3ee0 0x7f26bc0a63a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26d419cf20 0x7f26d419d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107290 0x7f26d4198ba0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 --2- 172.21.15.47:0/147642091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d41021c0 0x7f26d4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 -- 172.21.15.47:0/147642091 >> 172.21.15.47:0/147642091 conn(0x7f26d40fda20 msgr2=0x7f26d40fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.956+0000 7f26d92a9640 1 -- 172.21.15.47:0/147642091 shutdown_connections 2024-04-25T21:00:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:14.957+0000 7f26d92a9640 1 -- 172.21.15.47:0/147642091 wait complete. 2024-04-25T21:00:14.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:14 smithi186 ceph-mon[32529]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T21:00:15.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:14 smithi047 ceph-mon[29195]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T21:00:15.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:14 smithi092 ceph-mon[32524]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T21:00:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 -- 172.21.15.47:0/3910361406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44107c30 msgr2=0x7fde44108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 --2- 172.21.15.47:0/3910361406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44107c30 0x7fde44108010 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fde340099c0 tx=0x7fde34009cd0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 -- 172.21.15.47:0/3910361406 shutdown_connections 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 --2- 172.21.15.47:0/3910361406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44107c30 0x7fde44108010 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 --2- 172.21.15.47:0/3910361406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde44102ed0 0x7fde4410e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 --2- 172.21.15.47:0/3910361406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde44102510 0x7fde44102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 -- 172.21.15.47:0/3910361406 >> 172.21.15.47:0/3910361406 conn(0x7fde44076c60 msgr2=0x7fde44077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 -- 172.21.15.47:0/3910361406 shutdown_connections 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 -- 172.21.15.47:0/3910361406 wait complete. 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 Processor -- start 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.343+0000 7fde49559700 1 -- start start 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde49559700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde44102510 0x7fde441991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde49559700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44102ed0 0x7fde44199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde49559700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde44107c30 0x7fde4419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde49559700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fde44111c70 con 0x7fde44102ed0 2024-04-25T21:00:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde49559700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fde44111af0 con 0x7fde44107c30 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde49559700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fde44111df0 con 0x7fde44102510 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44102ed0 0x7fde44199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44102ed0 0x7fde44199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51494/0 (socket says 172.21.15.47:51494) 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 -- 172.21.15.47:0/2270388107 learned_addr learned my addr 172.21.15.47:0/2270388107 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde437fe700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde44107c30 0x7fde4419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde42ffd700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde44102510 0x7fde441991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 -- 172.21.15.47:0/2270388107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde44102510 msgr2=0x7fde441991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde44102510 0x7fde441991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 -- 172.21.15.47:0/2270388107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde44107c30 msgr2=0x7fde4419daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde44107c30 0x7fde4419daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 -- 172.21.15.47:0/2270388107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4419e1e0 con 0x7fde44102ed0 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.344+0000 7fde427fc700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44102ed0 0x7fde44199710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c920 tx=0x7fde3800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.345+0000 7fde2bfff700 1 -- 172.21.15.47:0/2270388107 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38004a30 con 0x7fde44102ed0 2024-04-25T21:00:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.345+0000 7fde2bfff700 1 -- 172.21.15.47:0/2270388107 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde380054d0 con 0x7fde44102ed0 2024-04-25T21:00:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.345+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4419e4d0 con 0x7fde44102ed0 2024-04-25T21:00:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.345+0000 7fde2bfff700 1 -- 172.21.15.47:0/2270388107 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38007c70 con 0x7fde44102ed0 2024-04-25T21:00:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.345+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde441a5d50 con 0x7fde44102ed0 2024-04-25T21:00:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.346+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde44069730 con 0x7fde44102ed0 2024-04-25T21:00:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.347+0000 7fde2bfff700 1 -- 172.21.15.47:0/2270388107 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7fde38004c60 con 0x7fde44102ed0 2024-04-25T21:00:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.347+0000 7fde2bfff700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fde2c0a3f50 0x7fde2c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.348+0000 7fde42ffd700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fde2c0a3f50 0x7fde2c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.348+0000 7fde2bfff700 1 -- 172.21.15.47:0/2270388107 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fde380d0960 con 0x7fde44102ed0 2024-04-25T21:00:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.348+0000 7fde42ffd700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fde2c0a3f50 0x7fde2c0a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fde34009990 tx=0x7fde3400b790 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:15.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.352+0000 7fde2bfff700 1 -- 172.21.15.47:0/2270388107 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde38014030 con 0x7fde44102ed0 2024-04-25T21:00:15.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.608+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde4419a530 con 0x7fde44102ed0 2024-04-25T21:00:15.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.609+0000 7fde2bfff700 1 -- 172.21.15.47:0/2270388107 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11853 (secure 0 0 0) 0x7fde3809cd60 con 0x7fde44102ed0 2024-04-25T21:00:15.610 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:00:15.610 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:59:57.345381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:00:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fde2c0a3f50 msgr2=0x7fde2c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fde2c0a3f50 0x7fde2c0a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fde34009990 tx=0x7fde3400b790 comp rx=0 tx=0).stop 2024-04-25T21:00:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44102ed0 msgr2=0x7fde44199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44102ed0 0x7fde44199710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c920 tx=0x7fde3800cc30 comp rx=0 tx=0).stop 2024-04-25T21:00:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 shutdown_connections 2024-04-25T21:00:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fde2c0a3f50 0x7fde2c0a6410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde44107c30 0x7fde4419daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde44102510 0x7fde441991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 --2- 172.21.15.47:0/2270388107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44102ed0 0x7fde44199710 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.612+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 >> 172.21.15.47:0/2270388107 conn(0x7fde44076c60 msgr2=0x7fde44101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.613+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 shutdown_connections 2024-04-25T21:00:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:15.613+0000 7fde49559700 1 -- 172.21.15.47:0/2270388107 wait complete. 2024-04-25T21:00:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-04-25T21:00:15.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:15 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2270388107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:15 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2270388107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:16.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:15 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2270388107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:16.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:00:16.516 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:16.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:16 smithi047 ceph-mon[29195]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-25T21:00:16.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:16 smithi186 ceph-mon[32529]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-25T21:00:17.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:16 smithi092 ceph-mon[32524]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.151+0000 7f39799e8700 1 -- 172.21.15.47:0/49242240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397406a750 msgr2=0x7f397406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.151+0000 7f39799e8700 1 --2- 172.21.15.47:0/49242240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397406a750 0x7f397406abb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3968009990 tx=0x7f3968009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.151+0000 7f39799e8700 1 -- 172.21.15.47:0/49242240 shutdown_connections 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.151+0000 7f39799e8700 1 --2- 172.21.15.47:0/49242240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397406b0f0 0x7f397410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.151+0000 7f39799e8700 1 --2- 172.21.15.47:0/49242240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3974108a30 0x7f3974108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.151+0000 7f39799e8700 1 --2- 172.21.15.47:0/49242240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397406a750 0x7f397406abb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.151+0000 7f39799e8700 1 -- 172.21.15.47:0/49242240 >> 172.21.15.47:0/49242240 conn(0x7f39740788f0 msgr2=0x7f3974078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.151+0000 7f39799e8700 1 -- 172.21.15.47:0/49242240 shutdown_connections 2024-04-25T21:00:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.152+0000 7f39799e8700 1 -- 172.21.15.47:0/49242240 wait complete. 2024-04-25T21:00:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.152+0000 7f39799e8700 1 Processor -- start 2024-04-25T21:00:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.152+0000 7f39799e8700 1 -- start start 2024-04-25T21:00:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39799e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397406a750 0x7f3974101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39799e8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397406b0f0 0x7f3974101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39799e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974108a30 0x7f3974102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39799e8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3974112c70 con 0x7f3974108a30 2024-04-25T21:00:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39799e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3974112af0 con 0x7f397406a750 2024-04-25T21:00:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39799e8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3974112df0 con 0x7f397406b0f0 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974108a30 0x7f3974102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974108a30 0x7f3974102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51514/0 (socket says 172.21.15.47:51514) 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 -- 172.21.15.47:0/187203073 learned_addr learned my addr 172.21.15.47:0/187203073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f3972ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397406a750 0x7f3974101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39727fc700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397406b0f0 0x7f3974101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 -- 172.21.15.47:0/187203073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397406b0f0 msgr2=0x7f3974101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397406b0f0 0x7f3974101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 -- 172.21.15.47:0/187203073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397406a750 msgr2=0x7f3974101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397406a750 0x7f3974101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 -- 172.21.15.47:0/187203073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39741a76a0 con 0x7f3974108a30 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39737fe700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974108a30 0x7f3974102490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f396c00b550 tx=0x7f396c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39789e6700 1 -- 172.21.15.47:0/187203073 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c015020 con 0x7f3974108a30 2024-04-25T21:00:18.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39789e6700 1 -- 172.21.15.47:0/187203073 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396c011ce0 con 0x7f3974108a30 2024-04-25T21:00:18.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.153+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39741a7990 con 0x7f3974108a30 2024-04-25T21:00:18.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.154+0000 7f39789e6700 1 -- 172.21.15.47:0/187203073 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c010c90 con 0x7f3974108a30 2024-04-25T21:00:18.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.154+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39741a7ea0 con 0x7f3974108a30 2024-04-25T21:00:18.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.155+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3974104170 con 0x7f3974108a30 2024-04-25T21:00:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.156+0000 7f39789e6700 1 -- 172.21.15.47:0/187203073 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f396c018020 con 0x7f3974108a30 2024-04-25T21:00:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.156+0000 7f39789e6700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f395c0a3ed0 0x7f395c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.157+0000 7f3972ffd700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f395c0a3ed0 0x7f395c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.157+0000 7f39789e6700 1 -- 172.21.15.47:0/187203073 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f396c0d0550 con 0x7f3974108a30 2024-04-25T21:00:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.157+0000 7f3972ffd700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f395c0a3ed0 0x7f395c0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3964009550 tx=0x7f39640091b0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:18.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.160+0000 7f39789e6700 1 -- 172.21.15.47:0/187203073 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f396c01f020 con 0x7f3974108a30 2024-04-25T21:00:18.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.368+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3974102ef0 con 0x7f3974108a30 2024-04-25T21:00:18.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.370+0000 7f39789e6700 1 -- 172.21.15.47:0/187203073 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22974 (secure 0 0 0) 0x7f396c09cac0 con 0x7f3974108a30 2024-04-25T21:00:18.371 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:00:18.371 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":92,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T20:59:59.007784+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1752629947":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/831854467":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/2517449296":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6801/253800519":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6800/253800519":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/3318702222":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:00:18.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f395c0a3ed0 msgr2=0x7f395c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:18.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f395c0a3ed0 0x7f395c0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3964009550 tx=0x7f39640091b0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974108a30 msgr2=0x7f3974102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974108a30 0x7f3974102490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f396c00b550 tx=0x7f396c00b860 comp rx=0 tx=0).stop 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 shutdown_connections 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f395c0a3ed0 0x7f395c0a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974108a30 0x7f3974102490 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397406a750 0x7f3974101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 --2- 172.21.15.47:0/187203073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397406b0f0 0x7f3974101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 >> 172.21.15.47:0/187203073 conn(0x7f39740788f0 msgr2=0x7f39740ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 shutdown_connections 2024-04-25T21:00:18.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:18.372+0000 7f39799e8700 1 -- 172.21.15.47:0/187203073 wait complete. 2024-04-25T21:00:18.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:18 smithi186 ceph-mon[32529]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-04-25T21:00:18.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:18 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/187203073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:00:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:18 smithi092 ceph-mon[32524]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-04-25T21:00:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:18 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/187203073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:00:19.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:18 smithi047 ceph-mon[29195]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-04-25T21:00:19.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:18 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/187203073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:00:19.075 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-25T21:00:19.331 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:20.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:20 smithi186 ceph-mon[32529]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T21:00:21.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:20 smithi092 ceph-mon[32524]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T21:00:21.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:20 smithi047 ceph-mon[29195]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T21:00:21.621 INFO:teuthology.orchestra.run.smithi047.stdout:{"run_dir":"/var/run/ceph"} 2024-04-25T21:00:21.621 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-25T21:00:21.622 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 2919 2024-04-25T21:00:21.622 DEBUG:tasks.quiescer:Yielding 2024-04-25T21:00:21.622 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T21:00:21.637 INFO:tasks.workunit:Pulling workunits from ref e221b808d285b9d02e54cc10c53c84188e8e41e3 2024-04-25T21:00:21.637 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T21:00:21.637 INFO:tasks.workunit:timeout=3h 2024-04-25T21:00:21.637 INFO:tasks.workunit:cleanup=True 2024-04-25T21:00:21.638 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T21:00:21.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:00:21.657 INFO:teuthology.orchestra.run.smithi047.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T21:00:21.657 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T21:00:21.657 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T21:00:21.657 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T21:00:21.658 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T21:00:21.658 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-25 20:57:24.344227089 +0000 2024-04-25T21:00:21.658 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-25 20:57:24.344227089 +0000 2024-04-25T21:00:21.658 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-25 20:59:56.601071725 +0000 2024-04-25T21:00:21.658 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-25 20:57:24.344227089 +0000 2024-04-25T21:00:21.658 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T21:00:21.658 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T21:00:21.731 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e221b808d285b9d02e54cc10c53c84188e8e41e3 2024-04-25T21:00:21.787 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T21:00:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:22.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:22 smithi186 ceph-mon[32529]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T21:00:23.022 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:22 smithi047 ceph-mon[29195]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T21:00:23.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:22 smithi092 ceph-mon[32524]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T21:00:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 -- 172.21.15.47:0/1933699528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103950 msgr2=0x7fa95006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 --2- 172.21.15.47:0/1933699528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103950 0x7fa95006b110 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa938009990 tx=0x7fa938009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 -- 172.21.15.47:0/1933699528 shutdown_connections 2024-04-25T21:00:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 --2- 172.21.15.47:0/1933699528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103950 0x7fa95006b110 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 --2- 172.21.15.47:0/1933699528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95010f390 0x7fa950111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 --2- 172.21.15.47:0/1933699528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa95006b650 0x7fa95006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 -- 172.21.15.47:0/1933699528 >> 172.21.15.47:0/1933699528 conn(0x7fa9500788f0 msgr2=0x7fa950078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 -- 172.21.15.47:0/1933699528 shutdown_connections 2024-04-25T21:00:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.572+0000 7fa954864700 1 -- 172.21.15.47:0/1933699528 wait complete. 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.573+0000 7fa954864700 1 Processor -- start 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.573+0000 7fa954864700 1 -- start start 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.573+0000 7fa954864700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa95006b650 0x7fa95019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.573+0000 7fa954864700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa950103950 0x7fa95019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.573+0000 7fa954864700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa95010f390 0x7fa95019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.573+0000 7fa954864700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa950114e00 con 0x7fa95010f390 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.573+0000 7fa954864700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa950114c80 con 0x7fa95006b650 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.573+0000 7fa954864700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa950114f80 con 0x7fa950103950 2024-04-25T21:00:23.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa95010f390 0x7fa95019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa95010f390 0x7fa95019fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51526/0 (socket says 172.21.15.47:51526) 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 -- 172.21.15.47:0/3546359202 learned_addr learned my addr 172.21.15.47:0/3546359202 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94e59c700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa95006b650 0x7fa95019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94dd9b700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa950103950 0x7fa95019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 -- 172.21.15.47:0/3546359202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa950103950 msgr2=0x7fa95019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa950103950 0x7fa95019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 -- 172.21.15.47:0/3546359202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa95006b650 msgr2=0x7fa95019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa95006b650 0x7fa95019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 -- 172.21.15.47:0/3546359202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9501a02c0 con 0x7fa95010f390 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa94ed9d700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa95010f390 0x7fa95019fb80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa9480094f0 tx=0x7fa94800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa9477fe700 1 -- 172.21.15.47:0/3546359202 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948004db0 con 0x7fa95010f390 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa9477fe700 1 -- 172.21.15.47:0/3546359202 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa948009e20 con 0x7fa95010f390 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa9477fe700 1 -- 172.21.15.47:0/3546359202 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948015400 con 0x7fa95010f390 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9501a05b0 con 0x7fa95010f390 2024-04-25T21:00:23.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.574+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9501a7e30 con 0x7fa95010f390 2024-04-25T21:00:23.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.575+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa950101f80 con 0x7fa95010f390 2024-04-25T21:00:23.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.577+0000 7fa9477fe700 1 -- 172.21.15.47:0/3546359202 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7fa948005260 con 0x7fa95010f390 2024-04-25T21:00:23.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.577+0000 7fa9477fe700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa93c0a3ed0 0x7fa93c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:23.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.577+0000 7fa94e59c700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa93c0a3ed0 0x7fa93c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:23.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.577+0000 7fa9477fe700 1 -- 172.21.15.47:0/3546359202 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fa9480d0c50 con 0x7fa95010f390 2024-04-25T21:00:23.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.578+0000 7fa94e59c700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa93c0a3ed0 0x7fa93c0a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b5c0 tx=0x7fa938005960 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:23.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.581+0000 7fa9477fe700 1 -- 172.21.15.47:0/3546359202 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa948010070 con 0x7fa95010f390 2024-04-25T21:00:23.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.844+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9501a8000 con 0x7fa95010f390 2024-04-25T21:00:23.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.845+0000 7fa9477fe700 1 -- 172.21.15.47:0/3546359202 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11853 (secure 0 0 0) 0x7fa94809d0b0 con 0x7fa95010f390 2024-04-25T21:00:23.846 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:00:23.846 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:59:57.345381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:00:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa93c0a3ed0 msgr2=0x7fa93c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa93c0a3ed0 0x7fa93c0a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b5c0 tx=0x7fa938005960 comp rx=0 tx=0).stop 2024-04-25T21:00:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa95010f390 msgr2=0x7fa95019fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa95010f390 0x7fa95019fb80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa9480094f0 tx=0x7fa94800d3b0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 shutdown_connections 2024-04-25T21:00:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa93c0a3ed0 0x7fa93c0a6390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa95010f390 0x7fa95019fb80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa95006b650 0x7fa95019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 --2- 172.21.15.47:0/3546359202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa950103950 0x7fa95019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 >> 172.21.15.47:0/3546359202 conn(0x7fa9500788f0 msgr2=0x7fa950100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.847+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 shutdown_connections 2024-04-25T21:00:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:23.848+0000 7fa954864700 1 -- 172.21.15.47:0/3546359202 wait complete. 2024-04-25T21:00:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-04-25T21:00:24.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T20:56:53.085968+0000', 'modified': '2024-04-25T20:59:57.345381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24467, 'mds_2': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/3860377657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 3860377657}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 3860377657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/4095064414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 4095064414}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 4095064414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6839/60292465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 60292465}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 60292465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.92:6835/2097108178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 2097108178}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 2097108178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/1984144918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1984144918}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1984144918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/941998037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 941998037}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 941998037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24452, 24467, 24488]} 2024-04-25T21:00:24.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T21:00:24.532 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.e status 2024-04-25T21:00:24.824 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:24.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:24 smithi047 ceph-mon[29195]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T21:00:24.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:24 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3546359202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:24.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:24 smithi186 ceph-mon[32529]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T21:00:24.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:24 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3546359202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:25.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:24 smithi092 ceph-mon[32524]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T21:00:25.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:24 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3546359202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 -- 172.21.15.47:0/2196772977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8101940 msgr2=0x7f85b8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 --2- 172.21.15.47:0/2196772977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8101940 0x7f85b8101dc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f85a80098f0 tx=0x7f85a8009c00 comp rx=0 tx=0).stop 2024-04-25T21:00:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 -- 172.21.15.47:0/2196772977 shutdown_connections 2024-04-25T21:00:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 --2- 172.21.15.47:0/2196772977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8102300 0x7f85b810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 --2- 172.21.15.47:0/2196772977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85b8107990 0x7f85b8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 --2- 172.21.15.47:0/2196772977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8101940 0x7f85b8101dc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 -- 172.21.15.47:0/2196772977 >> 172.21.15.47:0/2196772977 conn(0x7f85b80788f0 msgr2=0x7f85b8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 -- 172.21.15.47:0/2196772977 shutdown_connections 2024-04-25T21:00:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.498+0000 7f85be3b2700 1 -- 172.21.15.47:0/2196772977 wait complete. 2024-04-25T21:00:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85be3b2700 1 Processor -- start 2024-04-25T21:00:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85be3b2700 1 -- start start 2024-04-25T21:00:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85be3b2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85b8101940 0x7f85b8199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85be3b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8102300 0x7f85b819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85be3b2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8107990 0x7f85b81a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85be3b2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8111ae0 con 0x7f85b8107990 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85be3b2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8111960 con 0x7f85b8102300 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85be3b2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8111c60 con 0x7f85b8101940 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85bc94f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8107990 0x7f85b81a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85bc94f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8107990 0x7f85b81a5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39960/0 (socket says 172.21.15.47:39960) 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85bc94f700 1 -- 172.21.15.47:0/1826057716 learned_addr learned my addr 172.21.15.47:0/1826057716 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85b77fe700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8102300 0x7f85b819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.499+0000 7f85b7fff700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85b8101940 0x7f85b8199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85bc94f700 1 -- 172.21.15.47:0/1826057716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85b8101940 msgr2=0x7f85b8199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85bc94f700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85b8101940 0x7f85b8199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85bc94f700 1 -- 172.21.15.47:0/1826057716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8102300 msgr2=0x7f85b819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85bc94f700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8102300 0x7f85b819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85bc94f700 1 -- 172.21.15.47:0/1826057716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b81a5580 con 0x7f85b8107990 2024-04-25T21:00:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85bc94f700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8107990 0x7f85b81a5040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f85b000c110 tx=0x7f85b000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85b57fa700 1 -- 172.21.15.47:0/1826057716 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00092e0 con 0x7f85b8107990 2024-04-25T21:00:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85b57fa700 1 -- 172.21.15.47:0/1826057716 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b0004bb0 con 0x7f85b8107990 2024-04-25T21:00:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85be3b2700 1 -- 172.21.15.47:0/1826057716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b81a5750 con 0x7f85b8107990 2024-04-25T21:00:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85b57fa700 1 -- 172.21.15.47:0/1826057716 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0007ce0 con 0x7f85b8107990 2024-04-25T21:00:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.500+0000 7f85be3b2700 1 -- 172.21.15.47:0/1826057716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81a5c60 con 0x7f85b8107990 2024-04-25T21:00:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.502+0000 7f85b57fa700 1 -- 172.21.15.47:0/1826057716 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f85b001a020 con 0x7f85b8107990 2024-04-25T21:00:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.503+0000 7f85b57fa700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f85a40a3f50 0x7f85a40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.503+0000 7f85b7fff700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f85a40a3f50 0x7f85a40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.503+0000 7f85b57fa700 1 -- 172.21.15.47:0/1826057716 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f85b00d0570 con 0x7f85b8107990 2024-04-25T21:00:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.503+0000 7f85b7fff700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f85a40a3f50 0x7f85a40a6410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009bd0 tx=0x7f85a0009260 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85c8c3bb80 1 Processor -- start 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85c8c3bb80 1 -- start start 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e1500 0x5573ed1e5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5573ed12ece0 con 0x5573ed1e5990 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573ed0d5910 con 0x5573ed1e0be0 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5573ed12aca0 con 0x5573ed1e1500 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85bc94f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85bc94f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39974/0 (socket says 172.21.15.47:39974) 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85bc94f700 1 -- 172.21.15.47:0/1594259175 learned_addr learned my addr 172.21.15.47:0/1594259175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85b77fe700 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e1500 0x5573ed1e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85b7fff700 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.504+0000 7f85bc94f700 1 -- 172.21.15.47:0/1594259175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e1500 msgr2=0x5573ed1e5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85bc94f700 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e1500 0x5573ed1e5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85bc94f700 1 -- 172.21.15.47:0/1594259175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 msgr2=0x5573ed1e0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85bc94f700 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1e0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85bc94f700 1 -- 172.21.15.47:0/1594259175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ed07c880 con 0x5573ed1e5990 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85b7fff700 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85bc94f700 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e5df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f85b0098ca0 tx=0x7f85b00985f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2b3b0097b3618c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85ae7fc700 1 -- 172.21.15.47:0/1594259175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b009be10 con 0x5573ed1e5990 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85ae7fc700 1 -- 172.21.15.47:0/1594259175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b009c9d0 con 0x5573ed1e5990 2024-04-25T21:00:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.505+0000 7f85ae7fc700 1 -- 172.21.15.47:0/1594259175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0004d60 con 0x5573ed1e5990 2024-04-25T21:00:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1594259175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 msgr2=0x5573ed1e5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e5df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f85b0098ca0 tx=0x7f85b00985f0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1594259175 shutdown_connections 2024-04-25T21:00:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e5df0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1594259175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e1500 0x5573ed1e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1594259175 >> 172.21.15.47:0/1594259175 conn(0x5573ed109640 msgr2=0x5573ed1d7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1594259175 shutdown_connections 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1594259175 wait complete. 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 -- start start 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.506+0000 7f85c8c3bb80 1 Processor -- start 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e1500 0x5573ed1ea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed1ea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1eda20 con 0x5573ed1e1500 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573ed0cdb80 con 0x5573ed1e0be0 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1edba0 con 0x5573ed1e5990 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85c8c3bb80 10 client.0 _ll_register_callbacks cb 0x5573ed0c7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e1500 0x5573ed1ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e1500 0x5573ed1ea150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39978/0 (socket says 172.21.15.47:39978) 2024-04-25T21:00:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 -- 172.21.15.47:0/2264802676 learned_addr learned my addr 172.21.15.47:0/2264802676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85bc94f700 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed1ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 -- 172.21.15.47:0/2264802676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 msgr2=0x5573ed1ea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed1ea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 -- 172.21.15.47:0/2264802676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 msgr2=0x5573ed1ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 -- 172.21.15.47:0/2264802676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ed0e6220 con 0x5573ed1e1500 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b7fff700 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85b77fe700 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e1500 0x5573ed1ea150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009230 tx=0x7f85a800f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa35f4884fbd0318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.507+0000 7f85977fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85977fe700 1 -- 172.21.15.47:0/2264802676 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8005240 con 0x5573ed1e1500 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85977fe700 1 -- 172.21.15.47:0/2264802676 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f85a80053e0 con 0x5573ed1e1500 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/2264802676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b009c9d0 con 0x5573ed1e1500 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85977fe700 1 -- 172.21.15.47:0/2264802676 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85a8030050 con 0x5573ed1e1500 2024-04-25T21:00:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 10 client.15306 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/2264802676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5573ed1f8280 con 0x5573ed1e1500 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85977fe700 1 -- 172.21.15.47:0/2264802676 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b009c9d0 con 0x5573ed1e1500 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85977fe700 1 -- 172.21.15.47:0/2264802676 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f85a802b030 con 0x5573ed1e1500 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 10 client.15306 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 10 client.15306 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 20 client.15306 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x5573ed1fa8f0 0x5573ed1fbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 4 client.15306 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.508+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/2264802676 --> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573ed13c4d0 con 0x5573ed1fa8f0 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.509+0000 7f85b7fff700 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x5573ed1fa8f0 0x5573ed1fbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.509+0000 7f85b7fff700 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x5573ed1fa8f0 0x5573ed1fbd60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c9255742409bd53 server_cookie=ba8d9c826408d1c5 in_seq=0 out_seq=0 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.509+0000 7f85977fe700 10 client.15306 ms_handle_connect on v2:172.21.15.92:6834/2097108178 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85977fe700 1 -- 172.21.15.47:0/2264802676 <== mds.2 v2:172.21.15.92:6834/2097108178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573ed13c4d0 con 0x5573ed1fa8f0 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85977fe700 10 client.15306 handle_command_reply: tid=0 2024-04-25T21:00:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 client.15306 shutdown 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/2264802676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e1500 msgr2=0x5573ed1ea150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e1500 0x5573ed1ea150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009230 tx=0x7f85a800f700 comp rx=0 tx=0).stop 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/2264802676 shutdown_connections 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x5573ed1fa8f0 0x5573ed1fbd60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed1ea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85977fe700 0 client.15306 ms_handle_reset on v2:172.21.15.92:6834/2097108178 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e0be0 0x5573ed1ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/2264802676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e1500 0x5573ed1ea150 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/2264802676 >> 172.21.15.47:0/2264802676 conn(0x5573ed109640 msgr2=0x5573ed1e6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/2264802676 shutdown_connections 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/2264802676 wait complete. 2024-04-25T21:00:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.510+0000 7f85c8c3bb80 20 client.15306 trim_cache size 0 max 16384 2024-04-25T21:00:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85c8c3bb80 1 Processor -- start 2024-04-25T21:00:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85c8c3bb80 1 -- start start 2024-04-25T21:00:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e0be0 0x5573ed1e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1fa0a0 con 0x5573ed1e5990 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1fa770 con 0x5573ed1e1500 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1fae40 con 0x5573ed1e0be0 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85bc94f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85bc94f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39990/0 (socket says 172.21.15.47:39990) 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85bc94f700 1 -- 172.21.15.47:0/3093101188 learned_addr learned my addr 172.21.15.47:0/3093101188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85b7fff700 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e0be0 0x5573ed1e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85b77fe700 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85bc94f700 1 -- 172.21.15.47:0/3093101188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e0be0 msgr2=0x5573ed1e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85bc94f700 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e0be0 0x5573ed1e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85bc94f700 1 -- 172.21.15.47:0/3093101188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 msgr2=0x5573ed1ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85bc94f700 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.532+0000 7f85bc94f700 1 -- 172.21.15.47:0/3093101188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ed1edb50 con 0x5573ed1e5990 2024-04-25T21:00:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.533+0000 7f85bc94f700 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e74e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f85b0098730 tx=0x7f85b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa1b49312261a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.533+0000 7f85adffb700 1 -- 172.21.15.47:0/3093101188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b009bc30 con 0x5573ed1e5990 2024-04-25T21:00:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.533+0000 7f85adffb700 1 -- 172.21.15.47:0/3093101188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b009c9d0 con 0x5573ed1e5990 2024-04-25T21:00:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.533+0000 7f85adffb700 1 -- 172.21.15.47:0/3093101188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00d00d0 con 0x5573ed1e5990 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/3093101188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 msgr2=0x5573ed1e74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e74e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f85b0098730 tx=0x7f85b0009da0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/3093101188 shutdown_connections 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e5990 0x5573ed1e74e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e0be0 0x5573ed1e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/3093101188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/3093101188 >> 172.21.15.47:0/3093101188 conn(0x5573ed109640 msgr2=0x5573ed1eb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/3093101188 shutdown_connections 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/3093101188 wait complete. 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.534+0000 7f85c8c3bb80 1 -- start start 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85c8c3bb80 1 Processor -- start 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e0be0 0x5573ed1e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85c8c3bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed201870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1fbbe0 con 0x5573ed1e0be0 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1fb510 con 0x5573ed1e1500 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85c8c3bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1fc2b0 con 0x5573ed1e5990 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e0be0 0x5573ed1e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85c8c3bb80 10 client.0 _ll_register_callbacks cb 0x5573ed0c7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e0be0 0x5573ed1e9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40002/0 (socket says 172.21.15.47:40002) 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85b7fff700 1 -- 172.21.15.47:0/1017400616 learned_addr learned my addr 172.21.15.47:0/1017400616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85b77fe700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85bc94f700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed201870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.535+0000 7f85b7fff700 1 -- 172.21.15.47:0/1017400616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 msgr2=0x5573ed201870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85b7fff700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed201870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85b7fff700 1 -- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 msgr2=0x5573ed1ea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85b7fff700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85b7fff700 1 -- 172.21.15.47:0/1017400616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ed201de0 con 0x5573ed1e0be0 2024-04-25T21:00:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85bc94f700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed201870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85b77fe700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85b7fff700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e0be0 0x5573ed1e9d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097b0 tx=0x7f85a0013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3412f6c2c1fc4df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85977fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.536+0000 7f85977fe700 1 -- 172.21.15.47:0/1017400616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a002c070 con 0x5573ed1e0be0 2024-04-25T21:00:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1017400616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b009c9d0 con 0x5573ed1e0be0 2024-04-25T21:00:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85977fe700 1 -- 172.21.15.47:0/1017400616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f85a0028440 con 0x5573ed1e0be0 2024-04-25T21:00:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85977fe700 1 -- 172.21.15.47:0/1017400616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85a0034050 con 0x5573ed1e0be0 2024-04-25T21:00:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85c8c3bb80 10 client.15312 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1017400616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5573ed262310 con 0x5573ed1e0be0 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85977fe700 1 -- 172.21.15.47:0/1017400616 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b009c9d0 con 0x5573ed1e0be0 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85977fe700 1 -- 172.21.15.47:0/1017400616 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f85a002f030 con 0x5573ed1e0be0 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85c8c3bb80 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85c8c3bb80 10 client.15312 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85c8c3bb80 20 client.15312 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.537+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x5573ed1fa8f0 0x5573ed265cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.538+0000 7f85c8c3bb80 4 client.15312 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.538+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1017400616 --> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] -- command(tid 0: {"prefix": "status"}) -- 0x5573ed136570 con 0x5573ed1fa8f0 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.538+0000 7f85b77fe700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x5573ed1fa8f0 0x5573ed265cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.538+0000 7f85b77fe700 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x5573ed1fa8f0 0x5573ed265cc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83ea61039167285e server_cookie=684f73c5cf373ad4 in_seq=0 out_seq=0 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.538+0000 7f85977fe700 10 client.15312 ms_handle_connect on v2:172.21.15.92:6834/2097108178 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.538+0000 7f85977fe700 1 -- 172.21.15.47:0/1017400616 <== mds.2 v2:172.21.15.92:6834/2097108178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5573ed136570 con 0x5573ed1fa8f0 2024-04-25T21:00:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.538+0000 7f85977fe700 10 client.15312 handle_command_reply: tid=0 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.538+0000 7f85c8c3bb80 1 client.15312 shutdown 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1017400616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e0be0 msgr2=0x5573ed1e9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e0be0 0x5573ed1e9d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097b0 tx=0x7f85a0013070 comp rx=0 tx=0).stop 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1017400616 shutdown_connections 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x5573ed1fa8f0 0x5573ed265cc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ed1e0be0 0x5573ed1e9d90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573ed1e5990 0x5573ed201870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 --2- 172.21.15.47:0/1017400616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573ed1e1500 0x5573ed1ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1017400616 >> 172.21.15.47:0/1017400616 conn(0x5573ed109640 msgr2=0x5573ed1eb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85977fe700 0 client.15312 ms_handle_reset on v2:172.21.15.92:6834/2097108178 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1017400616 shutdown_connections 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 1 -- 172.21.15.47:0/1017400616 wait complete. 2024-04-25T21:00:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.539+0000 7f85c8c3bb80 20 client.15312 trim_cache size 0 max 16384 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24452, 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 199.82460127300001, 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 26, 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 92, 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 249.95456899800001 2024-04-25T21:00:26.544 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:00:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 -- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f85a40a3f50 msgr2=0x7f85a40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f85a40a3f50 0x7f85a40a6410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009bd0 tx=0x7f85a0009260 comp rx=0 tx=0).stop 2024-04-25T21:00:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 -- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8107990 msgr2=0x7f85b81a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8107990 0x7f85b81a5040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f85b000c110 tx=0x7f85b000c420 comp rx=0 tx=0).stop 2024-04-25T21:00:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 -- 172.21.15.47:0/1826057716 shutdown_connections 2024-04-25T21:00:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f85a40a3f50 0x7f85a40a6410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8107990 0x7f85b81a5040 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85b8101940 0x7f85b8199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 --2- 172.21.15.47:0/1826057716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8102300 0x7f85b819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 -- 172.21.15.47:0/1826057716 >> 172.21.15.47:0/1826057716 conn(0x7f85b80788f0 msgr2=0x7f85b80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 -- 172.21.15.47:0/1826057716 shutdown_connections 2024-04-25T21:00:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:26.540+0000 7f85be3b2700 1 -- 172.21.15.47:0/1826057716 wait complete. 2024-04-25T21:00:26.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:26 smithi047 ceph-mon[29195]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T21:00:27.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:26 smithi092 ceph-mon[32524]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T21:00:27.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:26 smithi186 ceph-mon[32529]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T21:00:27.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T21:00:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 -- 172.21.15.47:0/1611405936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 msgr2=0x7fced010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 --2- 172.21.15.47:0/1611405936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 0x7fced010ee60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009ca0 tx=0x7fcec002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 -- 172.21.15.47:0/1611405936 shutdown_connections 2024-04-25T21:00:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 --2- 172.21.15.47:0/1611405936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 0x7fced010ee60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 --2- 172.21.15.47:0/1611405936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fced0102b70 0x7fced0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 --2- 172.21.15.47:0/1611405936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced01021c0 0x7fced01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 -- 172.21.15.47:0/1611405936 >> 172.21.15.47:0/1611405936 conn(0x7fced00fda20 msgr2=0x7fced00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 -- 172.21.15.47:0/1611405936 shutdown_connections 2024-04-25T21:00:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 -- 172.21.15.47:0/1611405936 wait complete. 2024-04-25T21:00:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.046+0000 7fced5736640 1 Processor -- start 2024-04-25T21:00:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fced5736640 1 -- start start 2024-04-25T21:00:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fced5736640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fced01021c0 0x7fced0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fced5736640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 0x7fced0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fced5736640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced0074cd0 0x7fced01a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fced5736640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fced01121e0 con 0x7fced0107290 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fced5736640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fced0112060 con 0x7fced01021c0 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fced5736640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fced0112360 con 0x7fced0074cd0 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcece7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 0x7fced0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcece7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 0x7fced0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40018/0 (socket says 172.21.15.47:40018) 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcece7fc640 1 -- 172.21.15.47:0/2876604872 learned_addr learned my addr 172.21.15.47:0/2876604872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fceceffd640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fced01021c0 0x7fced0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced0074cd0 0x7fced01a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcece7fc640 1 -- 172.21.15.47:0/2876604872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced0074cd0 msgr2=0x7fced01a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcece7fc640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced0074cd0 0x7fced01a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcece7fc640 1 -- 172.21.15.47:0/2876604872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fced01021c0 msgr2=0x7fced0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcece7fc640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fced01021c0 0x7fced0074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.047+0000 7fcece7fc640 1 -- 172.21.15.47:0/2876604872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced01a6e10 con 0x7fced0107290 2024-04-25T21:00:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.048+0000 7fcece7fc640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 0x7fced0074790 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fceb800c970 tx=0x7fceb800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.048+0000 7fcec7fff640 1 -- 172.21.15.47:0/2876604872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8014070 con 0x7fced0107290 2024-04-25T21:00:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.048+0000 7fced5736640 1 -- 172.21.15.47:0/2876604872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced01a7100 con 0x7fced0107290 2024-04-25T21:00:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.048+0000 7fcec7fff640 1 -- 172.21.15.47:0/2876604872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb8004440 con 0x7fced0107290 2024-04-25T21:00:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.048+0000 7fcec7fff640 1 -- 172.21.15.47:0/2876604872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8003d30 con 0x7fced0107290 2024-04-25T21:00:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.048+0000 7fced5736640 1 -- 172.21.15.47:0/2876604872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced01a7650 con 0x7fced0107290 2024-04-25T21:00:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.050+0000 7fcec7fff640 1 -- 172.21.15.47:0/2876604872 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7fceb8017070 con 0x7fced0107290 2024-04-25T21:00:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.050+0000 7fced8453740 1 Processor -- start 2024-04-25T21:00:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.050+0000 7fced8453740 1 -- start start 2024-04-25T21:00:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fced8453740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455620 0x5625b6455a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fced8453740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455f40 0x5625b64602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fced8453740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6460800 0x5625b6462bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fced8453740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5625b63ba970 con 0x5625b6460800 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fced8453740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5625b63574a0 con 0x5625b6455620 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fced8453740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5625b63b6930 con 0x5625b6455f40 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcece7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6460800 0x5625b6462bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcec7fff640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcebc0a3f10 0x7fcebc0a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455620 0x5625b6455a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455620 0x5625b6455a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:53480/0 (socket says 172.21.15.47:53480) 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 -- 172.21.15.47:0/3309222180 learned_addr learned my addr 172.21.15.47:0/3309222180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcece7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6460800 0x5625b6462bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40028/0 (socket says 172.21.15.47:40028) 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fceceffd640 1 --2- 172.21.15.47:0/3309222180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455f40 0x5625b64602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcebc0a3f10 0x7fcebc0a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 -- 172.21.15.47:0/3309222180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455f40 msgr2=0x5625b64602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/3309222180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455f40 0x5625b64602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 -- 172.21.15.47:0/3309222180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6460800 msgr2=0x5625b6462bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/3309222180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6460800 0x5625b6462bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcec7fff640 1 -- 172.21.15.47:0/2876604872 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fceb80d0590 con 0x7fced0107290 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.051+0000 7fcecf7fe640 1 -- 172.21.15.47:0/3309222180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b60cd210 con 0x5625b6455620 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.052+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/3309222180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455620 0x5625b6455a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcec0005cc0 tx=0x7fcec0031f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ea6898694afcf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.052+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcebc0a3f10 0x7fcebc0a63d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcec0047000 tx=0x7fcec0049000 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.052+0000 7fcec57fa640 1 -- 172.21.15.47:0/3309222180 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec00318f0 con 0x5625b6455620 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.052+0000 7fcec57fa640 1 -- 172.21.15.47:0/3309222180 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec0036040 con 0x5625b6455620 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.052+0000 7fcec57fa640 1 -- 172.21.15.47:0/3309222180 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec002fd50 con 0x5625b6455620 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 -- 172.21.15.47:0/3309222180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455620 msgr2=0x5625b6455a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 --2- 172.21.15.47:0/3309222180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455620 0x5625b6455a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcec0005cc0 tx=0x7fcec0031f30 comp rx=0 tx=0).stop 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 -- 172.21.15.47:0/3309222180 shutdown_connections 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 --2- 172.21.15.47:0/3309222180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6460800 0x5625b6462bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 --2- 172.21.15.47:0/3309222180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455f40 0x5625b64602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 --2- 172.21.15.47:0/3309222180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455620 0x5625b6455a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 -- 172.21.15.47:0/3309222180 >> 172.21.15.47:0/3309222180 conn(0x5625b6451b30 msgr2=0x5625b6451f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 -- 172.21.15.47:0/3309222180 shutdown_connections 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 -- 172.21.15.47:0/3309222180 wait complete. 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 -- start start 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 Processor -- start 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b64601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b645d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 0x5625b645daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.053+0000 7fced8453740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5625b6463370 con 0x5625b6455620 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fced8453740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5625b63550b0 con 0x5625b6455f40 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fced8453740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5625b64634f0 con 0x5625b6460800 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fced8453740 10 client.0 _ll_register_callbacks cb 0x5625b6342d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b64601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b64601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40044/0 (socket says 172.21.15.47:40044) 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 -- 172.21.15.47:0/1098285322 learned_addr learned my addr 172.21.15.47:0/1098285322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fcece7fc640 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b645d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 0x5625b645daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 -- 172.21.15.47:0/1098285322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 msgr2=0x5625b645daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 0x5625b645daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 -- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 msgr2=0x5625b645d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b645d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 -- 172.21.15.47:0/1098285322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b6199620 con 0x5625b6455620 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fcece7fc640 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b645d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceceffd640 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b64601f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcec8010a50 tx=0x7fcec8010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bcebcee1476cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceb67fc640 1 -- 172.21.15.47:0/1098285322 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec801e060 con 0x5625b6455620 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fceb67fc640 1 -- 172.21.15.47:0/1098285322 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fcec800fb10 con 0x5625b6455620 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.054+0000 7fced8453740 1 -- 172.21.15.47:0/1098285322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec0036040 con 0x5625b6455620 2024-04-25T21:00:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fceb67fc640 1 -- 172.21.15.47:0/1098285322 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcec800fd10 con 0x5625b6455620 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fced8453740 10 client.15324 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fced8453740 1 -- 172.21.15.47:0/1098285322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625b646f380 con 0x5625b6455620 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fceb67fc640 1 -- 172.21.15.47:0/1098285322 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec0036040 con 0x5625b6455620 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fceb67fc640 1 -- 172.21.15.47:0/1098285322 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcec8021430 con 0x5625b6455620 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fced8453740 10 client.15324 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fced8453740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fced8453740 20 client.15324 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fced8453740 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5625b6470840 0x5625b6472c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fced8453740 4 client.15324 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fced8453740 1 -- 172.21.15.47:0/1098285322 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625b63c8390 con 0x5625b6470840 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.055+0000 7fcece7fc640 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5625b6470840 0x5625b6472c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.056+0000 7fcece7fc640 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5625b6470840 0x5625b6472c30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d514ce1f3da3bd4 server_cookie=15a0a712f5ef5c54 in_seq=0 out_seq=0 2024-04-25T21:00:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.056+0000 7fceb67fc640 10 client.15324 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.056+0000 7fceb67fc640 1 -- 172.21.15.47:0/1098285322 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625b63c8390 con 0x5625b6470840 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.056+0000 7fceb67fc640 10 client.15324 handle_command_reply: tid=0 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.056+0000 7fced8453740 1 client.15324 shutdown 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.056+0000 7fced8453740 1 -- 172.21.15.47:0/1098285322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 msgr2=0x5625b64601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.056+0000 7fced8453740 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b64601f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcec8010a50 tx=0x7fcec8010f10 comp rx=0 tx=0).stop 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 1 -- 172.21.15.47:0/1098285322 shutdown_connections 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5625b6470840 0x5625b6472c30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 0x5625b645daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fceb67fc640 0 client.15324 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b645d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 1 --2- 172.21.15.47:0/1098285322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b64601f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 1 -- 172.21.15.47:0/1098285322 >> 172.21.15.47:0/1098285322 conn(0x5625b6451b30 msgr2=0x5625b64578b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 1 -- 172.21.15.47:0/1098285322 shutdown_connections 2024-04-25T21:00:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 1 -- 172.21.15.47:0/1098285322 wait complete. 2024-04-25T21:00:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.057+0000 7fced8453740 20 client.15324 trim_cache size 0 max 16384 2024-04-25T21:00:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.071+0000 7fced8453740 1 Processor -- start 2024-04-25T21:00:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fced8453740 1 -- start start 2024-04-25T21:00:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fced8453740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455620 0x5625b6451430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fced8453740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455f40 0x5625b6457730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fced8453740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6460800 0x5625b645fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fced8453740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fced01b24f0 con 0x5625b6455f40 2024-04-25T21:00:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fced8453740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fced01afdc0 con 0x5625b6460800 2024-04-25T21:00:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fced8453740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fced01adc30 con 0x5625b6455620 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcece7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455f40 0x5625b6457730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcece7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455f40 0x5625b6457730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40058/0 (socket says 172.21.15.47:40058) 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcece7fc640 1 -- 172.21.15.47:0/2224433565 learned_addr learned my addr 172.21.15.47:0/2224433565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6460800 0x5625b645fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fceceffd640 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455620 0x5625b6451430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcece7fc640 1 -- 172.21.15.47:0/2224433565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455620 msgr2=0x5625b6451430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcece7fc640 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455620 0x5625b6451430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcece7fc640 1 -- 172.21.15.47:0/2224433565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6460800 msgr2=0x5625b645fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcece7fc640 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6460800 0x5625b645fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.072+0000 7fcece7fc640 1 -- 172.21.15.47:0/2224433565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b645de60 con 0x5625b6455f40 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.073+0000 7fcece7fc640 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455f40 0x5625b6457730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fceb809bf90 tx=0x7fceb80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a951dc74557bd066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.073+0000 7fcec4ff9640 1 -- 172.21.15.47:0/2224433565 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb809c290 con 0x5625b6455f40 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.073+0000 7fcec4ff9640 1 -- 172.21.15.47:0/2224433565 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb80a0640 con 0x5625b6455f40 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.073+0000 7fcec4ff9640 1 -- 172.21.15.47:0/2224433565 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb80a4cd0 con 0x5625b6455f40 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.073+0000 7fced8453740 1 -- 172.21.15.47:0/2224433565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455f40 msgr2=0x5625b6457730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.073+0000 7fced8453740 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455f40 0x5625b6457730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fceb809bf90 tx=0x7fceb80045f0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 -- 172.21.15.47:0/2224433565 shutdown_connections 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6460800 0x5625b645fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455f40 0x5625b6457730 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 --2- 172.21.15.47:0/2224433565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6455620 0x5625b6451430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 -- 172.21.15.47:0/2224433565 >> 172.21.15.47:0/2224433565 conn(0x5625b6451b30 msgr2=0x5625b6457020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 -- 172.21.15.47:0/2224433565 shutdown_connections 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 -- 172.21.15.47:0/2224433565 wait complete. 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 -- start start 2024-04-25T21:00:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 Processor -- start 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b6468740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b6468c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 0x5625b6459d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fced01ae280 con 0x5625b6455620 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fced01af6f0 con 0x5625b6455f40 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fced01ae950 con 0x5625b6460800 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fced8453740 10 client.0 _ll_register_callbacks cb 0x7fcec8010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fceceffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b6468740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fceceffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b6468740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40064/0 (socket says 172.21.15.47:40064) 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fceceffd640 1 -- 172.21.15.47:0/583284148 learned_addr learned my addr 172.21.15.47:0/583284148 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fcecf7fe640 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 0x5625b6459d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.074+0000 7fcece7fc640 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b6468c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceceffd640 1 -- 172.21.15.47:0/583284148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 msgr2=0x5625b6459d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceceffd640 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 0x5625b6459d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceceffd640 1 -- 172.21.15.47:0/583284148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 msgr2=0x5625b6468c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceceffd640 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b6468c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceceffd640 1 -- 172.21.15.47:0/583284148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b645a510 con 0x5625b6455620 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceceffd640 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b6468740 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcec8021a40 tx=0x7fcec8021d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79add9dda2bf593b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceb67fc640 1 -- 172.21.15.47:0/583284148 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec800a340 con 0x5625b6455620 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceb67fc640 1 -- 172.21.15.47:0/583284148 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fcec800fb10 con 0x5625b6455620 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fced8453740 1 -- 172.21.15.47:0/583284148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb80a0640 con 0x5625b6455620 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceb67fc640 1 -- 172.21.15.47:0/583284148 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcec8039050 con 0x5625b6455620 2024-04-25T21:00:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fced8453740 10 client.15330 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fced8453740 1 -- 172.21.15.47:0/583284148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625b64d6960 con 0x5625b6455620 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.075+0000 7fceb67fc640 1 -- 172.21.15.47:0/583284148 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb80a0640 con 0x5625b6455620 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fceb67fc640 1 -- 172.21.15.47:0/583284148 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcec8004bd0 con 0x5625b6455620 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fced8453740 10 client.15330 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fced8453740 10 client.15330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fced8453740 20 client.15330 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fced8453740 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5625b64d7100 0x5625b64d9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fced8453740 4 client.15330 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fced8453740 1 -- 172.21.15.47:0/583284148 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625b63c2200 con 0x5625b64d7100 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fcece7fc640 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5625b64d7100 0x5625b64d9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fceb67fc640 10 client.15330 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.076+0000 7fcece7fc640 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5625b64d7100 0x5625b64d9520 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed956731a101eea5 server_cookie=d4b6bfb300ef6d6c in_seq=0 out_seq=0 2024-04-25T21:00:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fceb67fc640 1 -- 172.21.15.47:0/583284148 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625b63c2200 con 0x5625b64d7100 2024-04-25T21:00:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fceb67fc640 10 client.15330 handle_command_reply: tid=0 2024-04-25T21:00:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 client.15330 shutdown 2024-04-25T21:00:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 -- 172.21.15.47:0/583284148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 msgr2=0x5625b6468740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b6468740 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcec8021a40 tx=0x7fcec8021d70 comp rx=0 tx=0).stop 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 -- 172.21.15.47:0/583284148 shutdown_connections 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5625b64d7100 0x5625b64d9520 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625b6460800 0x5625b6459d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625b6455f40 0x5625b6468c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fceb67fc640 0 client.15330 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 --2- 172.21.15.47:0/583284148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625b6455620 0x5625b6468740 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.084+0000 7fced8453740 1 -- 172.21.15.47:0/583284148 >> 172.21.15.47:0/583284148 conn(0x5625b6451b30 msgr2=0x5625b64589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced8453740 1 -- 172.21.15.47:0/583284148 shutdown_connections 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced8453740 1 -- 172.21.15.47:0/583284148 wait complete. 2024-04-25T21:00:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced8453740 20 client.15330 trim_cache size 0 max 16384 2024-04-25T21:00:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:00:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:00:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:00:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:00:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 -- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcebc0a3f10 msgr2=0x7fcebc0a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcebc0a3f10 0x7fcebc0a63d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcec0047000 tx=0x7fcec0049000 comp rx=0 tx=0).stop 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 -- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 msgr2=0x7fced0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 0x7fced0074790 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fceb800c970 tx=0x7fceb800cd70 comp rx=0 tx=0).stop 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 -- 172.21.15.47:0/2876604872 shutdown_connections 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced0074cd0 0x7fced01a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fced0107290 0x7fced0074790 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcebc0a3f10 0x7fcebc0a63d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 --2- 172.21.15.47:0/2876604872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fced01021c0 0x7fced0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 -- 172.21.15.47:0/2876604872 >> 172.21.15.47:0/2876604872 conn(0x7fced00fda20 msgr2=0x7fced00fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 -- 172.21.15.47:0/2876604872 shutdown_connections 2024-04-25T21:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:28.085+0000 7fced5736640 1 -- 172.21.15.47:0/2876604872 wait complete. 2024-04-25T21:00:28.973 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:28 smithi047 ceph-mon[29195]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T21:00:29.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:28 smithi092 ceph-mon[32524]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1633544558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c108a70 msgr2=0x7fcb6c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1633544558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c108a70 0x7fcb6c108e50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009990 tx=0x7fcb5c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1633544558 shutdown_connections 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1633544558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c108a70 0x7fcb6c108e50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1633544558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb6c103460 0x7fcb6c10f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1633544558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb6c102ac0 0x7fcb6c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1633544558 >> 172.21.15.47:0/1633544558 conn(0x7fcb6c079bf0 msgr2=0x7fcb6c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1633544558 shutdown_connections 2024-04-25T21:00:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.211+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1633544558 wait complete. 2024-04-25T21:00:29.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb72a6b700 1 Processor -- start 2024-04-25T21:00:29.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb72a6b700 1 -- start start 2024-04-25T21:00:29.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb72a6b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c102ac0 0x7fcb6c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb72a6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb6c103460 0x7fcb6c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb72a6b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb6c108a70 0x7fcb6c19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb72a6b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c112c10 con 0x7fcb6c102ac0 2024-04-25T21:00:29.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb72a6b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c112a90 con 0x7fcb6c103460 2024-04-25T21:00:29.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb72a6b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c112d90 con 0x7fcb6c108a70 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c102ac0 0x7fcb6c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c102ac0 0x7fcb6c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40074/0 (socket says 172.21.15.47:40074) 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 -- 172.21.15.47:0/1194693481 learned_addr learned my addr 172.21.15.47:0/1194693481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb6c103460 0x7fcb6c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb71008700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb6c108a70 0x7fcb6c19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 -- 172.21.15.47:0/1194693481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb6c108a70 msgr2=0x7fcb6c19d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb6c108a70 0x7fcb6c19d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 -- 172.21.15.47:0/1194693481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb6c103460 msgr2=0x7fcb6c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb6c103460 0x7fcb6c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 -- 172.21.15.47:0/1194693481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c19e130 con 0x7fcb6c102ac0 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.212+0000 7fcb70807700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c102ac0 0x7fcb6c199120 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c000c00 tx=0x7fcb5c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.213+0000 7fcb69ffb700 1 -- 172.21.15.47:0/1194693481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c027070 con 0x7fcb6c102ac0 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.213+0000 7fcb69ffb700 1 -- 172.21.15.47:0/1194693481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb5c022470 con 0x7fcb6c102ac0 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.213+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1194693481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6c19e3c0 con 0x7fcb6c102ac0 2024-04-25T21:00:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.213+0000 7fcb69ffb700 1 -- 172.21.15.47:0/1194693481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c017650 con 0x7fcb6c102ac0 2024-04-25T21:00:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.213+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1194693481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6c1a5ca0 con 0x7fcb6c102ac0 2024-04-25T21:00:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.215+0000 7fcb69ffb700 1 -- 172.21.15.47:0/1194693481 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7fcb5c02a070 con 0x7fcb6c102ac0 2024-04-25T21:00:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.216+0000 7fcb69ffb700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcb540a3f50 0x7fcb540a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.216+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcb540a3f50 0x7fcb540a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.216+0000 7fcb69ffb700 1 -- 172.21.15.47:0/1194693481 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fcb5c0d1820 con 0x7fcb6c102ac0 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcb540a3f50 0x7fcb540a6410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c07a260 tx=0x7fcb60009260 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb7d2f4b80 1 Processor -- start 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb7d2f4b80 1 -- start start 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf311bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf316040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf316580 0x5626bf3169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5626bf263ce0 con 0x5626bf316580 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5626bf20a910 con 0x5626bf3120f0 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5626bf25fca0 con 0x5626bf3117d0 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb71008700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf316580 0x5626bf3169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb71008700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf316580 0x5626bf3169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40076/0 (socket says 172.21.15.47:40076) 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb71008700 1 -- 172.21.15.47:0/1908836860 learned_addr learned my addr 172.21.15.47:0/1908836860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf316040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.217+0000 7fcb70807700 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf311bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb71008700 1 -- 172.21.15.47:0/1908836860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 msgr2=0x5626bf311bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb71008700 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf311bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb71008700 1 -- 172.21.15.47:0/1908836860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 msgr2=0x5626bf316040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb71008700 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf316040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb71008700 1 -- 172.21.15.47:0/1908836860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626bf1b1880 con 0x5626bf316580 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb71008700 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf316580 0x5626bf3169e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009c80 tx=0x7fcb64009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430831b091cfcc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb52ffd700 1 -- 172.21.15.47:0/1908836860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64015470 con 0x5626bf316580 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb52ffd700 1 -- 172.21.15.47:0/1908836860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb6401a070 con 0x5626bf316580 2024-04-25T21:00:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.218+0000 7fcb52ffd700 1 -- 172.21.15.47:0/1908836860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64014970 con 0x5626bf316580 2024-04-25T21:00:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:28 smithi186 ceph-mon[32529]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1908836860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf316580 msgr2=0x5626bf3169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf316580 0x5626bf3169e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009c80 tx=0x7fcb64009f90 comp rx=0 tx=0).stop 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1908836860 shutdown_connections 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf316580 0x5626bf3169e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf311bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1908836860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf316040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1908836860 >> 172.21.15.47:0/1908836860 conn(0x5626bf23e640 msgr2=0x5626bf30c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1908836860 shutdown_connections 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1908836860 wait complete. 2024-04-25T21:00:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 -- start start 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.219+0000 7fcb7d2f4b80 1 Processor -- start 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf31daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf31ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf316580 0x5626bf31b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5626bf31e610 con 0x5626bf3120f0 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5626bf202b80 con 0x5626bf316580 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5626bf31e790 con 0x5626bf3117d0 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb7d2f4b80 10 client.0 _ll_register_callbacks cb 0x5626bf1fc900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb6bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf31ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb6bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf31ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40092/0 (socket says 172.21.15.47:40092) 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb70807700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf31daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb6bfff700 1 -- 172.21.15.47:0/526234969 learned_addr learned my addr 172.21.15.47:0/526234969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb71008700 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf316580 0x5626bf31b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb6bfff700 1 -- 172.21.15.47:0/526234969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 msgr2=0x5626bf31daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf31daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb6bfff700 1 -- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf316580 msgr2=0x5626bf31b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf316580 0x5626bf31b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb6bfff700 1 -- 172.21.15.47:0/526234969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626bf21b220 con 0x5626bf3120f0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb70807700 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf31daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.220+0000 7fcb71008700 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf316580 0x5626bf31b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf31ada0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcb60011fa0 tx=0x7fcb600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0307b991d46bd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb4bfff700 1 -- 172.21.15.47:0/526234969 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60003d40 con 0x5626bf3120f0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/526234969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6401a070 con 0x5626bf3120f0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb4bfff700 1 -- 172.21.15.47:0/526234969 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fcb6001d410 con 0x5626bf3120f0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb4bfff700 1 -- 172.21.15.47:0/526234969 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb6001d610 con 0x5626bf3120f0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb7d2f4b80 10 client.15342 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/526234969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626bf328f70 con 0x5626bf3120f0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.221+0000 7fcb4bfff700 1 -- 172.21.15.47:0/526234969 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb6401a070 con 0x5626bf3120f0 2024-04-25T21:00:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb4bfff700 1 -- 172.21.15.47:0/526234969 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcb60030760 con 0x5626bf3120f0 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb7d2f4b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb7d2f4b80 10 client.15342 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb7d2f4b80 20 client.15342 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5626bf32b590 0x5626bf32ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb7d2f4b80 4 client.15342 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/526234969 --> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626bf2714d0 con 0x5626bf32b590 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb70807700 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5626bf32b590 0x5626bf32ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb70807700 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5626bf32b590 0x5626bf32ca00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4617d1847351cec server_cookie=4be5bbd07f855038 in_seq=0 out_seq=0 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.222+0000 7fcb4bfff700 10 client.15342 ms_handle_connect on v2:172.21.15.92:6836/1984144918 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb4bfff700 1 -- 172.21.15.47:0/526234969 <== mds.1 v2:172.21.15.92:6836/1984144918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626bf2714d0 con 0x5626bf32b590 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb4bfff700 10 client.15342 handle_command_reply: tid=0 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 client.15342 shutdown 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/526234969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 msgr2=0x5626bf31ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf31ada0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcb60011fa0 tx=0x7fcb600047c0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/526234969 shutdown_connections 2024-04-25T21:00:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5626bf32b590 0x5626bf32ca00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf316580 0x5626bf31b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf3117d0 0x5626bf31daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/526234969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf31ada0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/526234969 >> 172.21.15.47:0/526234969 conn(0x5626bf23e640 msgr2=0x5626bf30ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb4bfff700 0 client.15342 ms_handle_reset on v2:172.21.15.92:6836/1984144918 2024-04-25T21:00:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/526234969 shutdown_connections 2024-04-25T21:00:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.223+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/526234969 wait complete. 2024-04-25T21:00:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.224+0000 7fcb7d2f4b80 20 client.15342 trim_cache size 0 max 16384 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb7d2f4b80 1 Processor -- start 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb7d2f4b80 1 -- start start 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3117d0 0x5626bf319d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf336250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 0x5626bf315650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5626bf32ad40 con 0x5626bf3120f0 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5626bf32b410 con 0x5626bf3117d0 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5626bf32bae0 con 0x5626bf316580 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.245+0000 7fcb6bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf336250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb6bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf336250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40098/0 (socket says 172.21.15.47:40098) 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb6bfff700 1 -- 172.21.15.47:0/1552767725 learned_addr learned my addr 172.21.15.47:0/1552767725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb70807700 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3117d0 0x5626bf319d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb71008700 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 0x5626bf315650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb6bfff700 1 -- 172.21.15.47:0/1552767725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 msgr2=0x5626bf315650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 0x5626bf315650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb6bfff700 1 -- 172.21.15.47:0/1552767725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3117d0 msgr2=0x5626bf319d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3117d0 0x5626bf319d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb6bfff700 1 -- 172.21.15.47:0/1552767725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626bf31e740 con 0x5626bf3120f0 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf336250 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcb600097c0 tx=0x7fcb60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211a01416029450a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb527fc700 1 -- 172.21.15.47:0/1552767725 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60004560 con 0x5626bf3120f0 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb527fc700 1 -- 172.21.15.47:0/1552767725 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb60030920 con 0x5626bf3120f0 2024-04-25T21:00:29.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.246+0000 7fcb527fc700 1 -- 172.21.15.47:0/1552767725 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6001dd50 con 0x5626bf3120f0 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1552767725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 msgr2=0x5626bf336250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf336250 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcb600097c0 tx=0x7fcb60005880 comp rx=0 tx=0).stop 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1552767725 shutdown_connections 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 0x5626bf315650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3117d0 0x5626bf319d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1552767725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3120f0 0x5626bf336250 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1552767725 >> 172.21.15.47:0/1552767725 conn(0x5626bf23e640 msgr2=0x5626bf314f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1552767725 shutdown_connections 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1552767725 wait complete. 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 -- start start 2024-04-25T21:00:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.247+0000 7fcb7d2f4b80 1 Processor -- start 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3117d0 0x5626bf317230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf317770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb7d2f4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 0x5626bf317cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5626bf32c880 con 0x5626bf3117d0 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5626bf32c1b0 con 0x5626bf3120f0 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb7d2f4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5626bf32cf50 con 0x5626bf316580 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb7d2f4b80 10 client.0 _ll_register_callbacks cb 0x5626bf1fc900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb70807700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3117d0 0x5626bf317230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb70807700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3117d0 0x5626bf317230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40104/0 (socket says 172.21.15.47:40104) 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb70807700 1 -- 172.21.15.47:0/1211608651 learned_addr learned my addr 172.21.15.47:0/1211608651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf317770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb71008700 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 0x5626bf317cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb6bfff700 1 -- 172.21.15.47:0/1211608651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 msgr2=0x5626bf317cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 0x5626bf317cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb6bfff700 1 -- 172.21.15.47:0/1211608651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3117d0 msgr2=0x5626bf317230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3117d0 0x5626bf317230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb6bfff700 1 -- 172.21.15.47:0/1211608651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626bf3184d0 con 0x5626bf3120f0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.248+0000 7fcb70807700 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3117d0 0x5626bf317230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb6bfff700 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf317770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5626bf306c00 tx=0x7fcb60030470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45b456936ea87b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb4bfff700 1 -- 172.21.15.47:0/1211608651 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6001d7b0 con 0x5626bf3120f0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb4bfff700 1 -- 172.21.15.47:0/1211608651 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fcb6001d950 con 0x5626bf3120f0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1211608651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5626bf26b570 con 0x5626bf3120f0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb4bfff700 1 -- 172.21.15.47:0/1211608651 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb6001db50 con 0x5626bf3120f0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb7d2f4b80 10 client.25034 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1211608651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626bf396ea0 con 0x5626bf3120f0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb4bfff700 1 -- 172.21.15.47:0/1211608651 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5626bf26b570 con 0x5626bf3120f0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb4bfff700 1 -- 172.21.15.47:0/1211608651 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcb60041400 con 0x5626bf3120f0 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.249+0000 7fcb7d2f4b80 10 client.25034 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 10 client.25034 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 20 client.25034 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5626bf39b560 0x5626bf39a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 4 client.25034 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1211608651 --> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] -- command(tid 0: {"prefix": "status"}) -- 0x5626bf0e4200 con 0x5626bf39b560 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb70807700 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5626bf39b560 0x5626bf39a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb4bfff700 10 client.25034 ms_handle_connect on v2:172.21.15.92:6836/1984144918 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb70807700 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5626bf39b560 0x5626bf39a270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37c1a8c63759195f server_cookie=49a6e0ec2f571a5d in_seq=0 out_seq=0 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb4bfff700 1 -- 172.21.15.47:0/1211608651 <== mds.1 v2:172.21.15.92:6836/1984144918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5626bf0e4200 con 0x5626bf39b560 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb4bfff700 10 client.25034 handle_command_reply: tid=0 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 1 client.25034 shutdown 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 msgr2=0x5626bf317770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf317770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5626bf306c00 tx=0x7fcb60030470 comp rx=0 tx=0).stop 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1211608651 shutdown_connections 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.250+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x5626bf39b560 0x5626bf39a270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626bf3117d0 0x5626bf317230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb4bfff700 0 client.25034 ms_handle_reset on v2:172.21.15.92:6836/1984144918 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5626bf316580 0x5626bf317cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb7d2f4b80 1 --2- 172.21.15.47:0/1211608651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626bf3120f0 0x5626bf317770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1211608651 >> 172.21.15.47:0/1211608651 conn(0x5626bf23e640 msgr2=0x5626bf3157d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1211608651 shutdown_connections 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb7d2f4b80 1 -- 172.21.15.47:0/1211608651 wait complete. 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb7d2f4b80 20 client.25034 trim_cache size 0 max 16384 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24467, 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T21:00:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T21:00:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T21:00:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 203.54796714299999, 2024-04-25T21:00:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 26, 2024-04-25T21:00:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 92, 2024-04-25T21:00:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T21:00:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 244.323573525 2024-04-25T21:00:29.257 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:00:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcb540a3f50 msgr2=0x7fcb540a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcb540a3f50 0x7fcb540a6410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c07a260 tx=0x7fcb60009260 comp rx=0 tx=0).stop 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c102ac0 msgr2=0x7fcb6c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c102ac0 0x7fcb6c199120 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c000c00 tx=0x7fcb5c009580 comp rx=0 tx=0).stop 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1194693481 shutdown_connections 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fcb540a3f50 0x7fcb540a6410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb6c102ac0 0x7fcb6c199120 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb6c108a70 0x7fcb6c19d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 --2- 172.21.15.47:0/1194693481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb6c103460 0x7fcb6c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.251+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1194693481 >> 172.21.15.47:0/1194693481 conn(0x7fcb6c079bf0 msgr2=0x7fcb6c100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.252+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1194693481 shutdown_connections 2024-04-25T21:00:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:29.252+0000 7fcb72a6b700 1 -- 172.21.15.47:0/1194693481 wait complete. 2024-04-25T21:00:29.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T21:00:30.217 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:31.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:30 smithi092 ceph-mon[32524]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T21:00:31.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:30 smithi047 ceph-mon[29195]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T21:00:31.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:30 smithi186 ceph-mon[32529]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T21:00:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 -- 172.21.15.47:0/273310683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 msgr2=0x7fc83810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/273310683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 0x7fc83810e7f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc83000b0a0 tx=0x7fc83000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 -- 172.21.15.47:0/273310683 shutdown_connections 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/273310683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 0x7fc83810e7f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/273310683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc838107990 0x7fc838107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/273310683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc838101940 0x7fc838101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 -- 172.21.15.47:0/273310683 >> 172.21.15.47:0/273310683 conn(0x7fc8380788f0 msgr2=0x7fc838078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 -- 172.21.15.47:0/273310683 shutdown_connections 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.932+0000 7fc83e3a1700 1 -- 172.21.15.47:0/273310683 wait complete. 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc83e3a1700 1 Processor -- start 2024-04-25T21:00:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc83e3a1700 1 -- start start 2024-04-25T21:00:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc83e3a1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc838101940 0x7fc83819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc83e3a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 0x7fc83819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc83e3a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc838107990 0x7fc83819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc83e3a1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc838111b00 con 0x7fc838102300 2024-04-25T21:00:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc83e3a1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc838111980 con 0x7fc838107990 2024-04-25T21:00:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc83e3a1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc838111c80 con 0x7fc838101940 2024-04-25T21:00:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 0x7fc83819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 0x7fc83819b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40130/0 (socket says 172.21.15.47:40130) 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.933+0000 7fc8377fe700 1 -- 172.21.15.47:0/903540054 learned_addr learned my addr 172.21.15.47:0/903540054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc837fff700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc838101940 0x7fc83819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8377fe700 1 -- 172.21.15.47:0/903540054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc838101940 msgr2=0x7fc83819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8377fe700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc838101940 0x7fc83819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8377fe700 1 -- 172.21.15.47:0/903540054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc838107990 msgr2=0x7fc83819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8377fe700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc838107990 0x7fc83819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8377fe700 1 -- 172.21.15.47:0/903540054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8381a02c0 con 0x7fc838102300 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc837fff700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc838101940 0x7fc83819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8377fe700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 0x7fc83819b7f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc82000d980 tx=0x7fc82000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8357fa700 1 -- 172.21.15.47:0/903540054 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820016070 con 0x7fc838102300 2024-04-25T21:00:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc83e3a1700 1 -- 172.21.15.47:0/903540054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8381a05b0 con 0x7fc838102300 2024-04-25T21:00:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8357fa700 1 -- 172.21.15.47:0/903540054 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8200044d0 con 0x7fc838102300 2024-04-25T21:00:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc8357fa700 1 -- 172.21.15.47:0/903540054 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820004d00 con 0x7fc838102300 2024-04-25T21:00:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.934+0000 7fc83e3a1700 1 -- 172.21.15.47:0/903540054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8381a7e30 con 0x7fc838102300 2024-04-25T21:00:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.936+0000 7fc8357fa700 1 -- 172.21.15.47:0/903540054 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7fc820004020 con 0x7fc838102300 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.937+0000 7fc8357fa700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc8240a3ed0 0x7fc8240a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.937+0000 7fc8357fa700 1 -- 172.21.15.47:0/903540054 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fc8200d0990 con 0x7fc838102300 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.937+0000 7fc837fff700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc8240a3ed0 0x7fc8240a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc837fff700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc8240a3ed0 0x7fc8240a6390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc828009c90 tx=0x7fc828009260 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc848c2ab80 1 Processor -- start 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc848c2ab80 1 -- start start 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2507d0 0x55900d250bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2510f0 0x55900d255040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 0x55900d2559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55900d1a2ce0 con 0x55900d2510f0 2024-04-25T21:00:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55900d149910 con 0x55900d255580 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55900d19eca0 con 0x55900d2507d0 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2510f0 0x55900d255040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2510f0 0x55900d255040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40146/0 (socket says 172.21.15.47:40146) 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.938+0000 7fc8377fe700 1 -- 172.21.15.47:0/2018756308 learned_addr learned my addr 172.21.15.47:0/2018756308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc83c93e700 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 0x55900d2559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc837fff700 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2507d0 0x55900d250bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc8377fe700 1 -- 172.21.15.47:0/2018756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2507d0 msgr2=0x55900d250bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc8377fe700 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2507d0 0x55900d250bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc8377fe700 1 -- 172.21.15.47:0/2018756308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 msgr2=0x55900d2559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc8377fe700 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 0x55900d2559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc8377fe700 1 -- 172.21.15.47:0/2018756308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900d0f0880 con 0x55900d2510f0 2024-04-25T21:00:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc837fff700 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2507d0 0x55900d250bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc8377fe700 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2510f0 0x55900d255040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc820098b50 tx=0x7fc82009c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491ca110c8b351dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc82e7fc700 1 -- 172.21.15.47:0/2018756308 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200870c0 con 0x55900d2510f0 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc82e7fc700 1 -- 172.21.15.47:0/2018756308 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82009cdf0 con 0x55900d2510f0 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.939+0000 7fc82e7fc700 1 -- 172.21.15.47:0/2018756308 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82009cbb0 con 0x55900d2510f0 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2018756308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2510f0 msgr2=0x55900d255040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2510f0 0x55900d255040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc820098b50 tx=0x7fc82009c050 comp rx=0 tx=0).stop 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2018756308 shutdown_connections 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 0x55900d2559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2507d0 0x55900d250bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2018756308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2510f0 0x55900d255040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2018756308 >> 172.21.15.47:0/2018756308 conn(0x55900d17d640 msgr2=0x55900d24b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2018756308 shutdown_connections 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2018756308 wait complete. 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.940+0000 7fc848c2ab80 1 -- start start 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc848c2ab80 1 Processor -- start 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2510f0 0x55900d259d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 0x55900d25a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55900d25d5b0 con 0x55900d2507d0 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55900d141b80 con 0x55900d255580 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55900d25d730 con 0x55900d2510f0 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc837fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc848c2ab80 10 client.0 _ll_register_callbacks cb 0x55900d13b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc837fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40154/0 (socket says 172.21.15.47:40154) 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc837fff700 1 -- 172.21.15.47:0/2208440172 learned_addr learned my addr 172.21.15.47:0/2208440172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc8377fe700 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2510f0 0x55900d259d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc83c93e700 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 0x55900d25a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc837fff700 1 -- 172.21.15.47:0/2208440172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2510f0 msgr2=0x55900d259d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.941+0000 7fc837fff700 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2510f0 0x55900d259d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc837fff700 1 -- 172.21.15.47:0/2208440172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 msgr2=0x55900d25a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc837fff700 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 0x55900d25a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc837fff700 1 -- 172.21.15.47:0/2208440172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900d15a220 con 0x55900d2507d0 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc837fff700 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25ca70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc8280059d0 tx=0x7fc8280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd560430bf061f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc8177fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc8177fe700 1 -- 172.21.15.47:0/2208440172 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82801e830 con 0x55900d2507d0 2024-04-25T21:00:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc8177fe700 1 -- 172.21.15.47:0/2208440172 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fc82801e9d0 con 0x55900d2507d0 2024-04-25T21:00:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2208440172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82009cdf0 con 0x55900d2507d0 2024-04-25T21:00:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc8177fe700 1 -- 172.21.15.47:0/2208440172 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc82801ebd0 con 0x55900d2507d0 2024-04-25T21:00:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc848c2ab80 10 client.15360 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.942+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2208440172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55900d267e20 con 0x55900d2507d0 2024-04-25T21:00:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc8177fe700 1 -- 172.21.15.47:0/2208440172 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82801ed70 con 0x55900d2507d0 2024-04-25T21:00:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc8177fe700 1 -- 172.21.15.47:0/2208440172 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc82801b450 con 0x55900d2507d0 2024-04-25T21:00:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc848c2ab80 10 client.15360 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:31.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:31 smithi047 ceph-mon[29195]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T21:00:31.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc848c2ab80 10 client.15360 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T21:00:31.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc848c2ab80 20 client.15360 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:31.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55900d26a480 0x55900d26b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc848c2ab80 4 client.15360 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:31.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2208440172 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc82009cdf0 con 0x55900d26a480 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.943+0000 7fc8377fe700 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55900d26a480 0x55900d26b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.944+0000 7fc8177fe700 10 client.15360 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.944+0000 7fc8377fe700 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55900d26a480 0x55900d26b8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca4005fab99449db server_cookie=4fa6f471755c648 in_seq=0 out_seq=0 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.944+0000 7fc8177fe700 1 -- 172.21.15.47:0/2208440172 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc82009cdf0 con 0x55900d26a480 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc8177fe700 10 client.15360 handle_command_reply: tid=0 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 client.15360 shutdown 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2208440172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 msgr2=0x55900d25ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25ca70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc8280059d0 tx=0x7fc8280047c0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2208440172 shutdown_connections 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55900d26a480 0x55900d26b8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25ca70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc8177fe700 0 client.15360 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d255580 0x55900d25a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2208440172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d2510f0 0x55900d259d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2208440172 >> 172.21.15.47:0/2208440172 conn(0x55900d17d640 msgr2=0x55900d2569e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2208440172 shutdown_connections 2024-04-25T21:00:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2208440172 wait complete. 2024-04-25T21:00:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.945+0000 7fc848c2ab80 20 client.15360 trim_cache size 0 max 16384 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.966+0000 7fc848c2ab80 1 Processor -- start 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.966+0000 7fc848c2ab80 1 -- start start 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.966+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.966+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.966+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 0x55900d257610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55900d269c30 con 0x55900d2507d0 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55900d26a300 con 0x55900d2510f0 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55900d26a9d0 con 0x55900d255580 2024-04-25T21:00:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40162/0 (socket says 172.21.15.47:40162) 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 -- 172.21.15.47:0/2483481908 learned_addr learned my addr 172.21.15.47:0/2483481908 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc83c93e700 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 0x55900d257610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc8377fe700 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 -- 172.21.15.47:0/2483481908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 msgr2=0x55900d257610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 0x55900d257610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 -- 172.21.15.47:0/2483481908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 msgr2=0x55900d2570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 -- 172.21.15.47:0/2483481908 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900d25d150 con 0x55900d2507d0 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc8377fe700 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc837fff700 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25bcb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc8280059a0 tx=0x7fc828010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f189158377a64c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc82dffb700 1 -- 172.21.15.47:0/2483481908 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82801bce0 con 0x55900d2507d0 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc82dffb700 1 -- 172.21.15.47:0/2483481908 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8280099c0 con 0x55900d2507d0 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.967+0000 7fc82dffb700 1 -- 172.21.15.47:0/2483481908 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82802f980 con 0x55900d2507d0 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2483481908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 msgr2=0x55900d25bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25bcb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc8280059a0 tx=0x7fc828010b70 comp rx=0 tx=0).stop 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2483481908 shutdown_connections 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d25bcb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 0x55900d257610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/2483481908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2483481908 >> 172.21.15.47:0/2483481908 conn(0x55900d17d640 msgr2=0x55900d25b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2483481908 shutdown_connections 2024-04-25T21:00:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 -- 172.21.15.47:0/2483481908 wait complete. 2024-04-25T21:00:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.968+0000 7fc848c2ab80 1 -- start start 2024-04-25T21:00:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc848c2ab80 1 Processor -- start 2024-04-25T21:00:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d274d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2752b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc848c2ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 0x55900d26ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55900d26b770 con 0x55900d2507d0 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55900d26b0a0 con 0x55900d2510f0 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc848c2ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55900d26be40 con 0x55900d255580 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc848c2ab80 10 client.0 _ll_register_callbacks cb 0x55900d13b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc837fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d274d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc837fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d274d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40178/0 (socket says 172.21.15.47:40178) 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc837fff700 1 -- 172.21.15.47:0/4135615506 learned_addr learned my addr 172.21.15.47:0/4135615506 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.969+0000 7fc8377fe700 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc837fff700 1 -- 172.21.15.47:0/4135615506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 msgr2=0x55900d26ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc837fff700 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 0x55900d26ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc837fff700 1 -- 172.21.15.47:0/4135615506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 msgr2=0x55900d2752b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc837fff700 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2752b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc837fff700 1 -- 172.21.15.47:0/4135615506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900d270760 con 0x55900d2507d0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc837fff700 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d274d70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc82801b870 tx=0x7fc82801b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18868faff2a8308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc8177fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc8177fe700 1 -- 172.21.15.47:0/4135615506 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82801bc70 con 0x55900d2507d0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc8177fe700 1 -- 172.21.15.47:0/4135615506 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fc82801e630 con 0x55900d2507d0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc848c2ab80 1 -- 172.21.15.47:0/4135615506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55900d1b04d0 con 0x55900d2507d0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc8177fe700 1 -- 172.21.15.47:0/4135615506 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc82804d050 con 0x55900d2507d0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc848c2ab80 10 client.15366 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.970+0000 7fc848c2ab80 1 -- 172.21.15.47:0/4135615506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55900d2d21d0 con 0x55900d2507d0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc8177fe700 1 -- 172.21.15.47:0/4135615506 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55900d1b04d0 con 0x55900d2507d0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc8177fe700 1 -- 172.21.15.47:0/4135615506 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc828004920 con 0x55900d2507d0 2024-04-25T21:00:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc848c2ab80 10 client.15366 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc848c2ab80 10 client.15366 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc848c2ab80 20 client.15366 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55900d26a480 0x55900d2d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc848c2ab80 4 client.15366 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc848c2ab80 1 -- 172.21.15.47:0/4135615506 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "status"}) -- 0x55900d1aa570 con 0x55900d26a480 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc8377fe700 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55900d26a480 0x55900d2d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc8377fe700 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55900d26a480 0x55900d2d56e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6796595b613117cd server_cookie=daebd5ce7697ea3d in_seq=0 out_seq=0 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.971+0000 7fc8177fe700 10 client.15366 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc8177fe700 1 -- 172.21.15.47:0/4135615506 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55900d1aa570 con 0x55900d26a480 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc8177fe700 10 client.15366 handle_command_reply: tid=0 2024-04-25T21:00:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 client.15366 shutdown 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 -- 172.21.15.47:0/4135615506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 msgr2=0x55900d274d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d274d70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc82801b870 tx=0x7fc82801b670 comp rx=0 tx=0).stop 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 -- 172.21.15.47:0/4135615506 shutdown_connections 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55900d26a480 0x55900d2d56e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55900d2507d0 0x55900d274d70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55900d255580 0x55900d26ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc8177fe700 0 client.15366 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 --2- 172.21.15.47:0/4135615506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55900d2510f0 0x55900d2752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 -- 172.21.15.47:0/4135615506 >> 172.21.15.47:0/4135615506 conn(0x55900d17d640 msgr2=0x55900d24fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 -- 172.21.15.47:0/4135615506 shutdown_connections 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 1 -- 172.21.15.47:0/4135615506 wait complete. 2024-04-25T21:00:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.972+0000 7fc848c2ab80 20 client.15366 trim_cache size 0 max 16384 2024-04-25T21:00:31.980 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:00:31.980 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "0ca8de06-0345-11ef-bc93-c7b262605968", 2024-04-25T21:00:31.980 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-25T21:00:31.980 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24488, 2024-04-25T21:00:31.980 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-25T21:00:31.980 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-25T21:00:31.980 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-25T21:00:31.980 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 218.69541541000001, 2024-04-25T21:00:31.981 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 26, 2024-04-25T21:00:31.981 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 92, 2024-04-25T21:00:31.981 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T21:00:31.981 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 234.675427028 2024-04-25T21:00:31.981 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:00:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 -- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc8240a3ed0 msgr2=0x7fc8240a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc8240a3ed0 0x7fc8240a6390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc828009c90 tx=0x7fc828009260 comp rx=0 tx=0).stop 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 -- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 msgr2=0x7fc83819b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 0x7fc83819b7f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc82000d980 tx=0x7fc82000dd40 comp rx=0 tx=0).stop 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 -- 172.21.15.47:0/903540054 shutdown_connections 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc8240a3ed0 0x7fc8240a6390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc838107990 0x7fc83819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc838101940 0x7fc83819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 --2- 172.21.15.47:0/903540054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc838102300 0x7fc83819b7f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 -- 172.21.15.47:0/903540054 >> 172.21.15.47:0/903540054 conn(0x7fc8380788f0 msgr2=0x7fc83806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 -- 172.21.15.47:0/903540054 shutdown_connections 2024-04-25T21:00:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:31.973+0000 7fc83e3a1700 1 -- 172.21.15.47:0/903540054 wait complete. 2024-04-25T21:00:32.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:31 smithi186 ceph-mon[32529]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T21:00:32.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:31 smithi092 ceph-mon[32524]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T21:00:32.632 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-25T21:00:34.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:34 smithi047 ceph-mon[29195]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T21:00:34.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:34 smithi186 ceph-mon[32529]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T21:00:34.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:34 smithi092 ceph-mon[32524]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T21:00:36.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:36 smithi047 ceph-mon[29195]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T21:00:36.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:36 smithi186 ceph-mon[32529]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T21:00:36.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:36 smithi092 ceph-mon[32524]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T21:00:38.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:38 smithi047 ceph-mon[29195]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T21:00:38.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:38 smithi186 ceph-mon[32529]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T21:00:38.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:38 smithi092 ceph-mon[32524]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T21:00:40.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:40 smithi047 ceph-mon[29195]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:00:40.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:40 smithi186 ceph-mon[32529]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:00:40.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:40 smithi092 ceph-mon[32524]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:00:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.185+0000 7f536edcc640 1 -- 172.21.15.47:0/4144746608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368107290 msgr2=0x7f536810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.185+0000 7f536edcc640 1 --2- 172.21.15.47:0/4144746608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368107290 0x7f536810ee60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5360009ca0 tx=0x7f536002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 -- 172.21.15.47:0/4144746608 shutdown_connections 2024-04-25T21:00:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 --2- 172.21.15.47:0/4144746608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368107290 0x7f536810ee60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 --2- 172.21.15.47:0/4144746608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5368102b70 0x7f5368106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 --2- 172.21.15.47:0/4144746608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53681021c0 0x7f53681025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 -- 172.21.15.47:0/4144746608 >> 172.21.15.47:0/4144746608 conn(0x7f53680fda20 msgr2=0x7f53680ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 -- 172.21.15.47:0/4144746608 shutdown_connections 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 -- 172.21.15.47:0/4144746608 wait complete. 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 Processor -- start 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.186+0000 7f536edcc640 1 -- start start 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536edcc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53681021c0 0x7f5368074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536edcc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5368107290 0x7f5368074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536edcc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368074cd0 0x7f53681a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536edcc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53681121e0 con 0x7f5368074cd0 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536edcc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5368112060 con 0x7f53681021c0 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536edcc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5368112360 con 0x7f5368107290 2024-04-25T21:00:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368074cd0 0x7f53681a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368074cd0 0x7f53681a6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41854/0 (socket says 172.21.15.47:41854) 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 -- 172.21.15.47:0/235726076 learned_addr learned my addr 172.21.15.47:0/235726076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536cb68640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53681021c0 0x7f5368074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f5367fff640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5368107290 0x7f5368074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 -- 172.21.15.47:0/235726076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5368107290 msgr2=0x7f5368074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5368107290 0x7f5368074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 -- 172.21.15.47:0/235726076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53681021c0 msgr2=0x7f5368074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53681021c0 0x7f5368074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 -- 172.21.15.47:0/235726076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53681a6e90 con 0x7f5368074cd0 2024-04-25T21:00:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536cb68640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53681021c0 0x7f5368074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536d369640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368074cd0 0x7f53681a6790 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f53600098d0 tx=0x7f53600317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f5365ffb640 1 -- 172.21.15.47:0/235726076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360003b70 con 0x7f5368074cd0 2024-04-25T21:00:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f5365ffb640 1 -- 172.21.15.47:0/235726076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5360003d10 con 0x7f5368074cd0 2024-04-25T21:00:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.187+0000 7f536edcc640 1 -- 172.21.15.47:0/235726076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53681a7120 con 0x7f5368074cd0 2024-04-25T21:00:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.188+0000 7f5365ffb640 1 -- 172.21.15.47:0/235726076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536002fa20 con 0x7f5368074cd0 2024-04-25T21:00:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.188+0000 7f536edcc640 1 -- 172.21.15.47:0/235726076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a7670 con 0x7f5368074cd0 2024-04-25T21:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.190+0000 7f5365ffb640 1 -- 172.21.15.47:0/235726076 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f536002fc00 con 0x7f5368074cd0 2024-04-25T21:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.190+0000 7f5371bb7740 1 Processor -- start 2024-04-25T21:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5371bb7740 1 -- start start 2024-04-25T21:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5365ffb640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53580a3f10 0x7f53580a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71620 0x55fbe8d71a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5371bb7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fbe8cd6970 con 0x55fbe8d71f40 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5371bb7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fbe8c734a0 con 0x55fbe8d71620 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5371bb7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fbe8cd2930 con 0x55fbe8d7c800 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5365ffb640 1 -- 172.21.15.47:0/235726076 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f53600f1f90 con 0x7f5368074cd0 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5367fff640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53580a3f10 0x7f53580a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5367fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536cb68640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71620 0x55fbe8d71a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536cb68640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71620 0x55fbe8d71a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:59206/0 (socket says 172.21.15.47:59206) 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536cb68640 1 -- 172.21.15.47:0/3643758170 learned_addr learned my addr 172.21.15.47:0/3643758170 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5367fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41858/0 (socket says 172.21.15.47:41858) 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536d369640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536cb68640 1 -- 172.21.15.47:0/3643758170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 msgr2=0x55fbe8d7ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536cb68640 1 --2- 172.21.15.47:0/3643758170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536cb68640 1 -- 172.21.15.47:0/3643758170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71f40 msgr2=0x55fbe8d7c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536cb68640 1 --2- 172.21.15.47:0/3643758170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f536cb68640 1 -- 172.21.15.47:0/3643758170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe89e9210 con 0x55fbe8d71620 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.191+0000 7f5367fff640 1 --2- 172.21.15.47:0/3643758170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.192+0000 7f5367fff640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53580a3f10 0x7f53580a63d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5368075ce0 tx=0x7f53540099e0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.192+0000 7f536cb68640 1 --2- 172.21.15.47:0/3643758170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71620 0x55fbe8d71a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f535c008c60 tx=0x7f535c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64830e889e53bc4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.192+0000 7f5352ffd640 1 -- 172.21.15.47:0/3643758170 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c014070 con 0x55fbe8d71620 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.192+0000 7f5352ffd640 1 -- 172.21.15.47:0/3643758170 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c0047a0 con 0x55fbe8d71620 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.192+0000 7f5352ffd640 1 -- 172.21.15.47:0/3643758170 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c010690 con 0x55fbe8d71620 2024-04-25T21:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 -- 172.21.15.47:0/3643758170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71620 msgr2=0x55fbe8d71a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 --2- 172.21.15.47:0/3643758170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71620 0x55fbe8d71a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f535c008c60 tx=0x7f535c00e720 comp rx=0 tx=0).stop 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 -- 172.21.15.47:0/3643758170 shutdown_connections 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 --2- 172.21.15.47:0/3643758170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 --2- 172.21.15.47:0/3643758170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 --2- 172.21.15.47:0/3643758170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71620 0x55fbe8d71a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 -- 172.21.15.47:0/3643758170 >> 172.21.15.47:0/3643758170 conn(0x55fbe8d6db30 msgr2=0x55fbe8d6df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 -- 172.21.15.47:0/3643758170 shutdown_connections 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 -- 172.21.15.47:0/3643758170 wait complete. 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 -- start start 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.193+0000 7f5371bb7740 1 Processor -- start 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d7c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d79480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d79a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5371bb7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fbe8d7f430 con 0x55fbe8d71620 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5371bb7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fbe8c710b0 con 0x55fbe8d71f40 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5371bb7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fbe8d7f5b0 con 0x55fbe8d7c800 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5371bb7740 10 client.0 _ll_register_callbacks cb 0x55fbe8c5ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d7c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d7c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41866/0 (socket says 172.21.15.47:41866) 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 -- 172.21.15.47:0/1571906731 learned_addr learned my addr 172.21.15.47:0/1571906731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5367fff640 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d79480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536d369640 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d79a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 -- 172.21.15.47:0/1571906731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 msgr2=0x55fbe8d79a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d79a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 -- 172.21.15.47:0/1571906731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 msgr2=0x55fbe8d79480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d79480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 -- 172.21.15.47:0/1571906731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe8ab5620 con 0x55fbe8d71620 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536d369640 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d79a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f536cb68640 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d7c190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f535c010a50 tx=0x7f535c010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1e38fe0be7ca26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5347fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5347fff640 1 -- 172.21.15.47:0/1571906731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c014030 con 0x55fbe8d71620 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5347fff640 1 -- 172.21.15.47:0/1571906731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f535c004050 con 0x55fbe8d71620 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.194+0000 7f5371bb7740 1 -- 172.21.15.47:0/1571906731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f535c0047a0 con 0x55fbe8d71620 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5347fff640 1 -- 172.21.15.47:0/1571906731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f535c008830 con 0x55fbe8d71620 2024-04-25T21:00:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5371bb7740 10 client.15378 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5371bb7740 1 -- 172.21.15.47:0/1571906731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fbe8d8b380 con 0x55fbe8d71620 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5347fff640 1 -- 172.21.15.47:0/1571906731 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c0047a0 con 0x55fbe8d71620 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5347fff640 1 -- 172.21.15.47:0/1571906731 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f535c023420 con 0x55fbe8d71620 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5371bb7740 10 client.15378 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5371bb7740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5371bb7740 20 client.15378 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5371bb7740 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55fbe8d8c730 0x55fbe8d8eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5371bb7740 4 client.15378 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5371bb7740 1 -- 172.21.15.47:0/1571906731 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbe8ce4390 con 0x55fbe8d8c730 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.195+0000 7f5367fff640 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55fbe8d8c730 0x55fbe8d8eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.196+0000 7f5347fff640 10 client.15378 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.196+0000 7f5367fff640 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55fbe8d8c730 0x55fbe8d8eb20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89a4c1ab18e494b server_cookie=952ddf6e989a8601 in_seq=0 out_seq=0 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5347fff640 1 -- 172.21.15.47:0/1571906731 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbe8ce4390 con 0x55fbe8d8c730 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5347fff640 10 client.15378 handle_command_reply: tid=0 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 client.15378 shutdown 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 -- 172.21.15.47:0/1571906731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 msgr2=0x55fbe8d7c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d7c190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f535c010a50 tx=0x7f535c010a80 comp rx=0 tx=0).stop 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 -- 172.21.15.47:0/1571906731 shutdown_connections 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55fbe8d8c730 0x55fbe8d8eb20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d7c800 0x55fbe8d79a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d79480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 --2- 172.21.15.47:0/1571906731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d7c190 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5347fff640 0 client.15378 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 -- 172.21.15.47:0/1571906731 >> 172.21.15.47:0/1571906731 conn(0x55fbe8d6db30 msgr2=0x55fbe8d738b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 -- 172.21.15.47:0/1571906731 shutdown_connections 2024-04-25T21:00:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 1 -- 172.21.15.47:0/1571906731 wait complete. 2024-04-25T21:00:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.197+0000 7f5371bb7740 20 client.15378 trim_cache size 0 max 16384 2024-04-25T21:00:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.212+0000 7f5371bb7740 1 Processor -- start 2024-04-25T21:00:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.212+0000 7f5371bb7740 1 -- start start 2024-04-25T21:00:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.212+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71620 0x55fbe8d882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.212+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d6cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.212+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.212+0000 7f5371bb7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53681b2520 con 0x55fbe8d7c800 2024-04-25T21:00:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.212+0000 7f5371bb7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53681afdf0 con 0x55fbe8d71f40 2024-04-25T21:00:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.212+0000 7f5371bb7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f53681adc60 con 0x55fbe8d71620 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41870/0 (socket says 172.21.15.47:41870) 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 -- 172.21.15.47:0/3588403216 learned_addr learned my addr 172.21.15.47:0/3588403216 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536cb68640 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71620 0x55fbe8d882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f5367fff640 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d6cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 -- 172.21.15.47:0/3588403216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71620 msgr2=0x55fbe8d882f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71620 0x55fbe8d882f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 -- 172.21.15.47:0/3588403216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 msgr2=0x55fbe8d6cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d6cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 -- 172.21.15.47:0/3588403216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe8d7ebf0 con 0x55fbe8d7c800 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f5367fff640 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d6cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f536d369640 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7bcc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5360030040 tx=0x7f5360009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c1865c27b29398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f53527fc640 1 -- 172.21.15.47:0/3588403216 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600c1070 con 0x55fbe8d7c800 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f53527fc640 1 -- 172.21.15.47:0/3588403216 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5360036070 con 0x55fbe8d7c800 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.213+0000 7f53527fc640 1 -- 172.21.15.47:0/3588403216 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600ba410 con 0x55fbe8d7c800 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 -- 172.21.15.47:0/3588403216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d7c800 msgr2=0x55fbe8d7bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7bcc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5360030040 tx=0x7f5360009650 comp rx=0 tx=0).stop 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 -- 172.21.15.47:0/3588403216 shutdown_connections 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d7c800 0x55fbe8d7bcc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d71f40 0x55fbe8d6cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 --2- 172.21.15.47:0/3588403216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71620 0x55fbe8d882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 -- 172.21.15.47:0/3588403216 >> 172.21.15.47:0/3588403216 conn(0x55fbe8d6db30 msgr2=0x55fbe8d6c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 -- 172.21.15.47:0/3588403216 shutdown_connections 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 -- 172.21.15.47:0/3588403216 wait complete. 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 -- start start 2024-04-25T21:00:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.214+0000 7f5371bb7740 1 Processor -- start 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d84c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f5371bb7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d7c800 0x55fbe8d80590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f5371bb7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53681ae2b0 con 0x55fbe8d71620 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f5371bb7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53681af720 con 0x55fbe8d7c800 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f5371bb7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f53681ae980 con 0x55fbe8d71f40 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f5371bb7740 10 client.0 _ll_register_callbacks cb 0x7f535c008ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f536cb68640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d84c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f536cb68640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d84c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41872/0 (socket says 172.21.15.47:41872) 2024-04-25T21:00:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f536cb68640 1 -- 172.21.15.47:0/314672571 learned_addr learned my addr 172.21.15.47:0/314672571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f5367fff640 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f536d369640 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d7c800 0x55fbe8d80590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f536cb68640 1 -- 172.21.15.47:0/314672571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71f40 msgr2=0x55fbe8d7fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f536cb68640 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f536cb68640 1 -- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d7c800 msgr2=0x55fbe8d80590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.215+0000 7f536cb68640 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d7c800 0x55fbe8d80590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f536cb68640 1 -- 172.21.15.47:0/314672571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe8d80dc0 con 0x55fbe8d71620 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f536d369640 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d7c800 0x55fbe8d80590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f536cb68640 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d84c80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e760 tx=0x7f535c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b415364ee908075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5347fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5347fff640 1 -- 172.21.15.47:0/314672571 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c014070 con 0x55fbe8d71620 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5347fff640 1 -- 172.21.15.47:0/314672571 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f535c023bf0 con 0x55fbe8d71620 2024-04-25T21:00:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 1 -- 172.21.15.47:0/314672571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5360036070 con 0x55fbe8d71620 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5347fff640 1 -- 172.21.15.47:0/314672571 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f535c039050 con 0x55fbe8d71620 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 10 client.15384 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 1 -- 172.21.15.47:0/314672571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fbe8df2b70 con 0x55fbe8d71620 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5347fff640 1 -- 172.21.15.47:0/314672571 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5360036070 con 0x55fbe8d71620 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5347fff640 1 -- 172.21.15.47:0/314672571 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f535c003a80 con 0x55fbe8d71620 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 10 client.15384 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 20 client.15384 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55fbe8df3280 0x55fbe8df5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 4 client.15384 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.216+0000 7f5371bb7740 1 -- 172.21.15.47:0/314672571 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fbe8cde200 con 0x55fbe8df3280 2024-04-25T21:00:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.217+0000 7f5367fff640 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55fbe8df3280 0x55fbe8df5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.217+0000 7f5347fff640 10 client.15384 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.217+0000 7f5367fff640 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55fbe8df3280 0x55fbe8df5670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f222d351d04ea8f server_cookie=7139783d2325b504 in_seq=0 out_seq=0 2024-04-25T21:00:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5347fff640 1 -- 172.21.15.47:0/314672571 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fbe8cde200 con 0x55fbe8df3280 2024-04-25T21:00:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5347fff640 10 client.15384 handle_command_reply: tid=0 2024-04-25T21:00:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 client.15384 shutdown 2024-04-25T21:00:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 -- 172.21.15.47:0/314672571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 msgr2=0x55fbe8d84c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d84c80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e760 tx=0x7f535c00eef0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 -- 172.21.15.47:0/314672571 shutdown_connections 2024-04-25T21:00:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55fbe8df3280 0x55fbe8df5670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fbe8d7c800 0x55fbe8d80590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbe8d71f40 0x55fbe8d7fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 --2- 172.21.15.47:0/314672571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbe8d71620 0x55fbe8d84c80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5347fff640 0 client.15384 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.223+0000 7f5371bb7740 1 -- 172.21.15.47:0/314672571 >> 172.21.15.47:0/314672571 conn(0x55fbe8d6db30 msgr2=0x55fbe8d749b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f5371bb7740 1 -- 172.21.15.47:0/314672571 shutdown_connections 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f5371bb7740 1 -- 172.21.15.47:0/314672571 wait complete. 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f5371bb7740 20 client.15384 trim_cache size 0 max 16384 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:00:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:00:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:00:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 -- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53580a3f10 msgr2=0x7f53580a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53580a3f10 0x7f53580a63d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5368075ce0 tx=0x7f53540099e0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 -- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368074cd0 msgr2=0x7f53681a6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368074cd0 0x7f53681a6790 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f53600098d0 tx=0x7f53600317b0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 -- 172.21.15.47:0/235726076 shutdown_connections 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5368074cd0 0x7f53681a6790 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5368107290 0x7f5368074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53580a3f10 0x7f53580a63d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 --2- 172.21.15.47:0/235726076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53681021c0 0x7f5368074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 -- 172.21.15.47:0/235726076 >> 172.21.15.47:0/235726076 conn(0x7f53680fda20 msgr2=0x7f53680fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 -- 172.21.15.47:0/235726076 shutdown_connections 2024-04-25T21:00:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:41.224+0000 7f536edcc640 1 -- 172.21.15.47:0/235726076 wait complete. 2024-04-25T21:00:42.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:42 smithi047 ceph-mon[29195]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-25T21:00:42.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:42 smithi186 ceph-mon[32529]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-25T21:00:42.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:42 smithi092 ceph-mon[32524]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-25T21:00:44.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:44 smithi047 ceph-mon[29195]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:00:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:44 smithi186 ceph-mon[32529]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:00:44.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:44 smithi092 ceph-mon[32524]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:00:46.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:46 smithi047 ceph-mon[29195]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T21:00:46.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:46 smithi186 ceph-mon[32529]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T21:00:46.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:46 smithi092 ceph-mon[32524]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T21:00:47.633 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T21:00:47.633 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.2 2024-04-25T21:00:47.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:00:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:48.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:48 smithi047 ceph-mon[29195]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T21:00:48.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:48 smithi186 ceph-mon[32529]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T21:00:48.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:48 smithi092 ceph-mon[32524]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T21:00:50.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:50 smithi047 ceph-mon[29195]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T21:00:50.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:50 smithi186 ceph-mon[32529]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T21:00:50.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:50 smithi092 ceph-mon[32524]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T21:00:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 -- 172.21.15.47:0/3139715303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 msgr2=0x7f14bc10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 --2- 172.21.15.47:0/3139715303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 0x7f14bc10f900 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b0a0 tx=0x7f14b400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 -- 172.21.15.47:0/3139715303 shutdown_connections 2024-04-25T21:00:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 --2- 172.21.15.47:0/3139715303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 0x7f14bc10f900 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 --2- 172.21.15.47:0/3139715303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14bc108a30 0x7f14bc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 --2- 172.21.15.47:0/3139715303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14bc06a750 0x7f14bc06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 -- 172.21.15.47:0/3139715303 >> 172.21.15.47:0/3139715303 conn(0x7f14bc0788f0 msgr2=0x7f14bc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 -- 172.21.15.47:0/3139715303 shutdown_connections 2024-04-25T21:00:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.269+0000 7f14c2752700 1 -- 172.21.15.47:0/3139715303 wait complete. 2024-04-25T21:00:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c2752700 1 Processor -- start 2024-04-25T21:00:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c2752700 1 -- start start 2024-04-25T21:00:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c2752700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14bc06a750 0x7f14bc101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c2752700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 0x7f14bc101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c2752700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14bc108a30 0x7f14bc102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c2752700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc112bd0 con 0x7f14bc06b0f0 2024-04-25T21:00:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c2752700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc112a50 con 0x7f14bc108a30 2024-04-25T21:00:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c2752700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc112d50 con 0x7f14bc06a750 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14bb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 0x7f14bc101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14bb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 0x7f14bc101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32996/0 (socket says 172.21.15.47:32996) 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14bb7fe700 1 -- 172.21.15.47:0/3940104482 learned_addr learned my addr 172.21.15.47:0/3940104482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14c0cef700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14bc108a30 0x7f14bc102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.270+0000 7f14bbfff700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14bc06a750 0x7f14bc101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14bb7fe700 1 -- 172.21.15.47:0/3940104482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14bc06a750 msgr2=0x7f14bc101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14bb7fe700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14bc06a750 0x7f14bc101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14bb7fe700 1 -- 172.21.15.47:0/3940104482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14bc108a30 msgr2=0x7f14bc102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14bb7fe700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14bc108a30 0x7f14bc102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14bb7fe700 1 -- 172.21.15.47:0/3940104482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc1a75c0 con 0x7f14bc06b0f0 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14bb7fe700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 0x7f14bc101f50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f14b000e9a0 tx=0x7f14b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14b97fa700 1 -- 172.21.15.47:0/3940104482 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b000cbd0 con 0x7f14bc06b0f0 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14b97fa700 1 -- 172.21.15.47:0/3940104482 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b000cd70 con 0x7f14bc06b0f0 2024-04-25T21:00:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc1a78b0 con 0x7f14bc06b0f0 2024-04-25T21:00:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14b97fa700 1 -- 172.21.15.47:0/3940104482 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0018680 con 0x7f14bc06b0f0 2024-04-25T21:00:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.271+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc1a7df0 con 0x7f14bc06b0f0 2024-04-25T21:00:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.272+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc069730 con 0x7f14bc06b0f0 2024-04-25T21:00:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.273+0000 7f14b97fa700 1 -- 172.21.15.47:0/3940104482 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f14b0010430 con 0x7f14bc06b0f0 2024-04-25T21:00:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.274+0000 7f14b97fa700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f14a40a3ed0 0x7f14a40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.274+0000 7f14b97fa700 1 -- 172.21.15.47:0/3940104482 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f14b00d1100 con 0x7f14bc06b0f0 2024-04-25T21:00:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.274+0000 7f14bbfff700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f14a40a3ed0 0x7f14a40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.275+0000 7f14bbfff700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f14a40a3ed0 0x7f14a40a6390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00dad0 tx=0x7f14ac00d4b0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.280+0000 7f14b97fa700 1 -- 172.21.15.47:0/3940104482 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14b009d600 con 0x7f14bc06b0f0 2024-04-25T21:00:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.578+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14bc02df80 con 0x7f14bc06b0f0 2024-04-25T21:00:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.579+0000 7f14b97fa700 1 -- 172.21.15.47:0/3940104482 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11853 (secure 0 0 0) 0x7f14b009cf00 con 0x7f14bc06b0f0 2024-04-25T21:00:51.580 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:00:51.580 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:59:57.345381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:00:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f14a40a3ed0 msgr2=0x7f14a40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f14a40a3ed0 0x7f14a40a6390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00dad0 tx=0x7f14ac00d4b0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 msgr2=0x7f14bc101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 0x7f14bc101f50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f14b000e9a0 tx=0x7f14b000ecb0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 shutdown_connections 2024-04-25T21:00:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f14a40a3ed0 0x7f14a40a6390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14bc108a30 0x7f14bc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14bc06a750 0x7f14bc101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 --2- 172.21.15.47:0/3940104482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14bc06b0f0 0x7f14bc101f50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.582+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 >> 172.21.15.47:0/3940104482 conn(0x7f14bc0788f0 msgr2=0x7f14bc0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.583+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 shutdown_connections 2024-04-25T21:00:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:51.583+0000 7f14c2752700 1 -- 172.21.15.47:0/3940104482 wait complete. 2024-04-25T21:00:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-04-25T21:00:52.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:52 smithi047 ceph-mon[29195]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T21:00:52.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:52 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3940104482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:52.396 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0' 2024-04-25T21:00:52.396 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24488 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0 2024-04-25T21:00:52.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:52 smithi186 ceph-mon[32529]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T21:00:52.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:52 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3940104482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:52.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:52 smithi092 ceph-mon[32524]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T21:00:52.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:52 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3940104482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 -- 172.21.15.47:0/3890987401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c101680 msgr2=0x7f803c10e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 --2- 172.21.15.47:0/3890987401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c101680 0x7f803c10e3d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8024009900 tx=0x7f802402a410 comp rx=0 tx=0).stop 2024-04-25T21:00:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 -- 172.21.15.47:0/3890987401 shutdown_connections 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 --2- 172.21.15.47:0/3890987401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f803c10e910 0x7f803c110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 --2- 172.21.15.47:0/3890987401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c101680 0x7f803c10e3d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 --2- 172.21.15.47:0/3890987401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803c100d60 0x7f803c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 -- 172.21.15.47:0/3890987401 >> 172.21.15.47:0/3890987401 conn(0x7f803c077fd0 msgr2=0x7f803c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 -- 172.21.15.47:0/3890987401 shutdown_connections 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.321+0000 7f8040a31640 1 -- 172.21.15.47:0/3890987401 wait complete. 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.322+0000 7f8040a31640 1 Processor -- start 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.322+0000 7f8040a31640 1 -- start start 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.322+0000 7f8040a31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803c100d60 0x7f803c103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.322+0000 7f8040a31640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f803c10e910 0x7f803c103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f8040a31640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c103ac0 0x7f803c1ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f8040a31640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1143d0 con 0x7f803c103ac0 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f8040a31640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f803c114250 con 0x7f803c100d60 2024-04-25T21:00:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f8040a31640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f803c114550 con 0x7f803c10e910 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803c100d60 0x7f803c103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803c100d60 0x7f803c103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:46402/0 (socket says 172.21.15.47:46402) 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 -- 172.21.15.47:0/977835468 learned_addr learned my addr 172.21.15.47:0/977835468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f8033fff640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f803c10e910 0x7f803c103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803ad9d640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c103ac0 0x7f803c1ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 -- 172.21.15.47:0/977835468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f803c10e910 msgr2=0x7f803c103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f803c10e910 0x7f803c103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 -- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c103ac0 msgr2=0x7f803c1ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c103ac0 0x7f803c1ab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 -- 172.21.15.47:0/977835468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c1ab5b0 con 0x7f803c100d60 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803ad9d640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c103ac0 0x7f803c1ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.323+0000 7f803a59c640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803c100d60 0x7f803c103040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c890 tx=0x7f802c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.324+0000 7f80337fe640 1 -- 172.21.15.47:0/977835468 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c004220 con 0x7f803c100d60 2024-04-25T21:00:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.324+0000 7f8040a31640 1 -- 172.21.15.47:0/977835468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c1ab8a0 con 0x7f803c100d60 2024-04-25T21:00:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.324+0000 7f8040a31640 1 -- 172.21.15.47:0/977835468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c1abe10 con 0x7f803c100d60 2024-04-25T21:00:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.324+0000 7f80337fe640 1 -- 172.21.15.47:0/977835468 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f802c0043c0 con 0x7f803c100d60 2024-04-25T21:00:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.324+0000 7f80337fe640 1 -- 172.21.15.47:0/977835468 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c01d060 con 0x7f803c100d60 2024-04-25T21:00:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.326+0000 7f80337fe640 1 -- 172.21.15.47:0/977835468 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f802c005020 con 0x7f803c100d60 2024-04-25T21:00:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f80337fe640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f80280a3ee0 0x7f80280a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8043829740 1 Processor -- start 2024-04-25T21:00:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f80337fe640 1 -- 172.21.15.47:0/977835468 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f802c0d0420 con 0x7f803c100d60 2024-04-25T21:00:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8043829740 1 -- start start 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8033fff640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f80280a3ee0 0x7f80280a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8043829740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 0x564cd6dd8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8043829740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8043829740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8043829740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564cd6d3d970 con 0x564cd6dd8f40 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8043829740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564cd6cda4a0 con 0x564cd6de3800 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f8043829740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564cd6d39930 con 0x564cd6dd8620 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f803ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f803ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:46406/0 (socket says 172.21.15.47:46406) 2024-04-25T21:00:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.327+0000 7f803ad9d640 1 -- 172.21.15.47:0/2964386965 learned_addr learned my addr 172.21.15.47:0/2964386965 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f803a59c640 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 0x564cd6dd8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f8033fff640 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f803ad9d640 1 -- 172.21.15.47:0/2964386965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 msgr2=0x564cd6dd8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f803ad9d640 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 0x564cd6dd8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f803ad9d640 1 -- 172.21.15.47:0/2964386965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 msgr2=0x564cd6de32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f803ad9d640 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f803ad9d640 1 -- 172.21.15.47:0/2964386965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cd6a50210 con 0x564cd6de3800 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f8033fff640 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f803ad9d640 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de5bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8034010720 tx=0x7f8034010be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b8e0819854876f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f8033fff640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f80280a3ee0 0x7f80280a63a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8024005940 tx=0x7f8024005b50 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f8030ff9640 1 -- 172.21.15.47:0/2964386965 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034004b20 con 0x564cd6de3800 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f8030ff9640 1 -- 172.21.15.47:0/2964386965 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803400ede0 con 0x564cd6de3800 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.328+0000 7f8030ff9640 1 -- 172.21.15.47:0/2964386965 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034005820 con 0x564cd6de3800 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 -- 172.21.15.47:0/2964386965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 msgr2=0x564cd6de5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de5bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8034010720 tx=0x7f8034010be0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 -- 172.21.15.47:0/2964386965 shutdown_connections 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de5bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 --2- 172.21.15.47:0/2964386965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 0x564cd6dd8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 -- 172.21.15.47:0/2964386965 >> 172.21.15.47:0/2964386965 conn(0x564cd6dd4b30 msgr2=0x564cd6dd4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 -- 172.21.15.47:0/2964386965 shutdown_connections 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.329+0000 7f8043829740 1 -- 172.21.15.47:0/2964386965 wait complete. 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 1 -- start start 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 1 Processor -- start 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 0x564cd6de31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564cd6de6430 con 0x564cd6dd8f40 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564cd6cd80b0 con 0x564cd6de3800 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564cd6de65b0 con 0x564cd6dd8620 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8043829740 10 client.0 _ll_register_callbacks cb 0x564cd6cc5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8033fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8033fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46204/0 (socket says 172.21.15.47:46204) 2024-04-25T21:00:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f8033fff640 1 -- 172.21.15.47:0/3091128705 learned_addr learned my addr 172.21.15.47:0/3091128705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f803ad9d640 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.330+0000 7f803a59c640 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 0x564cd6de31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8033fff640 1 -- 172.21.15.47:0/3091128705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 msgr2=0x564cd6de31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8033fff640 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 0x564cd6de31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8033fff640 1 -- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 msgr2=0x564cd6de0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8033fff640 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8033fff640 1 -- 172.21.15.47:0/3091128705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cd6b1c620 con 0x564cd6dd8f40 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f803ad9d640 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8033fff640 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de04b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8024032bb0 tx=0x7f802402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953282c4f9870087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8021ffb640 1 -- 172.21.15.47:0/3091128705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024035040 con 0x564cd6dd8f40 2024-04-25T21:00:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8021ffb640 1 -- 172.21.15.47:0/3091128705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f8024005040 con 0x564cd6dd8f40 2024-04-25T21:00:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8043829740 1 -- 172.21.15.47:0/3091128705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803400ede0 con 0x564cd6dd8f40 2024-04-25T21:00:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8021ffb640 1 -- 172.21.15.47:0/3091128705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8024059050 con 0x564cd6dd8f40 2024-04-25T21:00:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8043829740 10 client.15399 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.331+0000 7f8043829740 1 -- 172.21.15.47:0/3091128705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cd6df2330 con 0x564cd6dd8f40 2024-04-25T21:00:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f8021ffb640 1 -- 172.21.15.47:0/3091128705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803400ede0 con 0x564cd6dd8f40 2024-04-25T21:00:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f8021ffb640 1 -- 172.21.15.47:0/3091128705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8024004cb0 con 0x564cd6dd8f40 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f8043829740 10 client.15399 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f8043829740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f8043829740 20 client.15399 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f8043829740 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564cd6df3840 0x564cd6df5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f8043829740 4 client.15399 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f8043829740 1 -- 172.21.15.47:0/3091128705 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cd6d4b390 con 0x564cd6df3840 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.332+0000 7f803a59c640 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564cd6df3840 0x564cd6df5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.333+0000 7f8021ffb640 10 client.15399 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.333+0000 7f803a59c640 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564cd6df3840 0x564cd6df5c50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a145e0055fe7e95 server_cookie=f173ab0b7bfa1da1 in_seq=0 out_seq=0 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.333+0000 7f8021ffb640 1 -- 172.21.15.47:0/3091128705 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cd6d4b390 con 0x564cd6df3840 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.333+0000 7f8021ffb640 10 client.15399 handle_command_reply: tid=0 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.333+0000 7f8043829740 1 client.15399 shutdown 2024-04-25T21:00:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 -- 172.21.15.47:0/3091128705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 msgr2=0x564cd6de04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de04b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8024032bb0 tx=0x7f802402a410 comp rx=0 tx=0).stop 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 -- 172.21.15.47:0/3091128705 shutdown_connections 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564cd6df3840 0x564cd6df5c50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6de3800 0x564cd6de0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6dd8f40 0x564cd6de04b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8021ffb640 0 client.15399 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 --2- 172.21.15.47:0/3091128705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8620 0x564cd6de31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 -- 172.21.15.47:0/3091128705 >> 172.21.15.47:0/3091128705 conn(0x564cd6dd4b30 msgr2=0x564cd6dda8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 -- 172.21.15.47:0/3091128705 shutdown_connections 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 1 -- 172.21.15.47:0/3091128705 wait complete. 2024-04-25T21:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.334+0000 7f8043829740 20 client.15399 trim_cache size 0 max 16384 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.358+0000 7f8043829740 1 Processor -- start 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.358+0000 7f8043829740 1 -- start start 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.358+0000 7f8043829740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6def2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.358+0000 7f8043829740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6de2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.358+0000 7f8043829740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6dfb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.358+0000 7f8043829740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b7010 con 0x564cd6de3800 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.358+0000 7f8043829740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b48e0 con 0x564cd6dd8620 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.358+0000 7f8043829740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b2750 con 0x564cd6dd8f40 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6dfb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6dfb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46216/0 (socket says 172.21.15.47:46216) 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 -- 172.21.15.47:0/3910621095 learned_addr learned my addr 172.21.15.47:0/3910621095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803a59c640 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6def2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f8033fff640 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6de2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 -- 172.21.15.47:0/3910621095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 msgr2=0x564cd6de2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6de2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 -- 172.21.15.47:0/3910621095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 msgr2=0x564cd6def2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6def2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 -- 172.21.15.47:0/3910621095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cd6dd8d70 con 0x564cd6de3800 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f8033fff640 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6de2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f803ad9d640 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6dfb560 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f80340106c0 tx=0x7f8034013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fa9404c22dbbed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f8023fff640 1 -- 172.21.15.47:0/3910621095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034003b70 con 0x564cd6de3800 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f8023fff640 1 -- 172.21.15.47:0/3910621095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803400ede0 con 0x564cd6de3800 2024-04-25T21:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.359+0000 7f8023fff640 1 -- 172.21.15.47:0/3910621095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034005790 con 0x564cd6de3800 2024-04-25T21:00:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 -- 172.21.15.47:0/3910621095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 msgr2=0x564cd6dfb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6dfb560 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f80340106c0 tx=0x7f8034013040 comp rx=0 tx=0).stop 2024-04-25T21:00:54.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:54 smithi047 ceph-mon[29195]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T21:00:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 -- 172.21.15.47:0/3910621095 shutdown_connections 2024-04-25T21:00:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6dfb560 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6de2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 --2- 172.21.15.47:0/3910621095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6def2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 -- 172.21.15.47:0/3910621095 >> 172.21.15.47:0/3910621095 conn(0x564cd6dd4b30 msgr2=0x564cd6ddbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 -- 172.21.15.47:0/3910621095 shutdown_connections 2024-04-25T21:00:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.360+0000 7f8043829740 1 -- 172.21.15.47:0/3910621095 wait complete. 2024-04-25T21:00:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 1 -- start start 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 1 Processor -- start 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6deb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6debd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6de6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b2da0 con 0x564cd6de3800 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b4210 con 0x564cd6dd8620 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b3470 con 0x564cd6dd8f40 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8043829740 10 client.0 _ll_register_callbacks cb 0x7f8024032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f803ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6de6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f803ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6de6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46232/0 (socket says 172.21.15.47:46232) 2024-04-25T21:00:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f803ad9d640 1 -- 172.21.15.47:0/913380783 learned_addr learned my addr 172.21.15.47:0/913380783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f803a59c640 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6deb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.361+0000 7f8033fff640 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6debd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f803ad9d640 1 -- 172.21.15.47:0/913380783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 msgr2=0x564cd6debd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f803ad9d640 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6debd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f803ad9d640 1 -- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 msgr2=0x564cd6deb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f803ad9d640 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6deb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f803ad9d640 1 -- 172.21.15.47:0/913380783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cd6de7720 con 0x564cd6de3800 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f803a59c640 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6deb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f803ad9d640 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6de6ef0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8034005270 tx=0x7f80340052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6021ed45a63ad171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f8021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f8021ffb640 1 -- 172.21.15.47:0/913380783 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034003970 con 0x564cd6de3800 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f8021ffb640 1 -- 172.21.15.47:0/913380783 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f8034003b10 con 0x564cd6de3800 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f8043829740 1 -- 172.21.15.47:0/913380783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803400ede0 con 0x564cd6de3800 2024-04-25T21:00:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f8021ffb640 1 -- 172.21.15.47:0/913380783 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8034003d50 con 0x564cd6de3800 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f8043829740 10 client.15405 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.362+0000 7f8043829740 1 -- 172.21.15.47:0/913380783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cd6e59950 con 0x564cd6de3800 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8021ffb640 1 -- 172.21.15.47:0/913380783 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803400ede0 con 0x564cd6de3800 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8021ffb640 1 -- 172.21.15.47:0/913380783 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f803401a9e0 con 0x564cd6de3800 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8043829740 10 client.15405 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8043829740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8043829740 20 client.15405 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8043829740 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564cd6e59d70 0x564cd6e5c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8043829740 4 client.15405 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8043829740 1 -- 172.21.15.47:0/913380783 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564cd6d45200 con 0x564cd6e59d70 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f803a59c640 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564cd6e59d70 0x564cd6e5c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f8021ffb640 10 client.15405 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.363+0000 7f803a59c640 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564cd6e59d70 0x564cd6e5c160 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43b91ae14c6ec5c server_cookie=c193a71c0334f925 in_seq=0 out_seq=0 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8021ffb640 1 -- 172.21.15.47:0/913380783 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564cd6d45200 con 0x564cd6e59d70 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8021ffb640 10 client.15405 handle_command_reply: tid=0 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 client.15405 shutdown 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 -- 172.21.15.47:0/913380783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 msgr2=0x564cd6de6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6de6ef0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8034005270 tx=0x7f80340052a0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 -- 172.21.15.47:0/913380783 shutdown_connections 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564cd6e59d70 0x564cd6e5c160 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cd6de3800 0x564cd6de6ef0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cd6dd8f40 0x564cd6debd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 --2- 172.21.15.47:0/913380783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564cd6dd8620 0x564cd6deb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 -- 172.21.15.47:0/913380783 >> 172.21.15.47:0/913380783 conn(0x564cd6dd4b30 msgr2=0x564cd6dee2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8021ffb640 0 client.15405 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 -- 172.21.15.47:0/913380783 shutdown_connections 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 1 -- 172.21.15.47:0/913380783 wait complete. 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.367+0000 7f8043829740 20 client.15405 trim_cache size 0 max 16384 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:00:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:00:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 -- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f80280a3ee0 msgr2=0x7f80280a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f80280a3ee0 0x7f80280a63a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8024005940 tx=0x7f8024005b50 comp rx=0 tx=0).stop 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 -- 172.21.15.47:0/977835468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803c100d60 msgr2=0x7f803c103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803c100d60 0x7f803c103040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c890 tx=0x7f802c00cd50 comp rx=0 tx=0).stop 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 -- 172.21.15.47:0/977835468 shutdown_connections 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f803c103ac0 0x7f803c1ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f803c10e910 0x7f803c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f80280a3ee0 0x7f80280a63a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 --2- 172.21.15.47:0/977835468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803c100d60 0x7f803c103040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 -- 172.21.15.47:0/977835468 >> 172.21.15.47:0/977835468 conn(0x7f803c077fd0 msgr2=0x7f803c10ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 -- 172.21.15.47:0/977835468 shutdown_connections 2024-04-25T21:00:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:00:54.368+0000 7f8040a31640 1 -- 172.21.15.47:0/977835468 wait complete. 2024-04-25T21:00:54.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:54 smithi186 ceph-mon[32529]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T21:00:54.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:54 smithi092 ceph-mon[32524]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 -- 172.21.15.47:0/4226574672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370108a60 msgr2=0x7f8370108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 --2- 172.21.15.47:0/4226574672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370108a60 0x7f8370108e40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8364009990 tx=0x7f8364009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 -- 172.21.15.47:0/4226574672 shutdown_connections 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 --2- 172.21.15.47:0/4226574672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370108a60 0x7f8370108e40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 --2- 172.21.15.47:0/4226574672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f837006b100 0x7f837010f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 --2- 172.21.15.47:0/4226574672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837006a740 0x7f837006abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 -- 172.21.15.47:0/4226574672 >> 172.21.15.47:0/4226574672 conn(0x7f83700788e0 msgr2=0x7f8370078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 -- 172.21.15.47:0/4226574672 shutdown_connections 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.555+0000 7f83771fc700 1 -- 172.21.15.47:0/4226574672 wait complete. 2024-04-25T21:00:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f83771fc700 1 Processor -- start 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f83771fc700 1 -- start start 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f83771fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837006a740 0x7f8370198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f83771fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837006b100 0x7f8370199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f83771fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8370108a60 0x7f837019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f83771fc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8370112bf0 con 0x7f837006a740 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f83771fc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8370112a70 con 0x7f8370108a60 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f83771fc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8370112d70 con 0x7f837006b100 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f8374f98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837006a740 0x7f8370198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f8374f98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837006a740 0x7f8370198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46242/0 (socket says 172.21.15.47:46242) 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f8374f98700 1 -- 172.21.15.47:0/1135352331 learned_addr learned my addr 172.21.15.47:0/1135352331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f836ffff700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837006b100 0x7f8370199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f8374f98700 1 -- 172.21.15.47:0/1135352331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837006b100 msgr2=0x7f8370199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f8375799700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8370108a60 0x7f837019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f8374f98700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837006b100 0x7f8370199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f8374f98700 1 -- 172.21.15.47:0/1135352331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8370108a60 msgr2=0x7f837019d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.556+0000 7f8374f98700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8370108a60 0x7f837019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.557+0000 7f8374f98700 1 -- 172.21.15.47:0/1135352331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837019dfd0 con 0x7f837006a740 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.557+0000 7f836ffff700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837006b100 0x7f8370199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.557+0000 7f8374f98700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837006a740 0x7f8370198fc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f83640057c0 tx=0x7f83640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.557+0000 7f836dffb700 1 -- 172.21.15.47:0/1135352331 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364027070 con 0x7f837006a740 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.557+0000 7f836dffb700 1 -- 172.21.15.47:0/1135352331 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8364004020 con 0x7f837006a740 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.557+0000 7f83771fc700 1 -- 172.21.15.47:0/1135352331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837019e260 con 0x7f837006a740 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.557+0000 7f836dffb700 1 -- 172.21.15.47:0/1135352331 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364022c40 con 0x7f837006a740 2024-04-25T21:00:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.557+0000 7f83771fc700 1 -- 172.21.15.47:0/1135352331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8370101940 con 0x7f837006a740 2024-04-25T21:00:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.560+0000 7f836dffb700 1 -- 172.21.15.47:0/1135352331 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138873+0+0 (secure 0 0 0) 0x7f836402a070 con 0x7f837006a740 2024-04-25T21:00:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.560+0000 7f836dffb700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f83600a3f70 0x7f83600a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.560+0000 7f836dffb700 1 -- 172.21.15.47:0/1135352331 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f83640d1a20 con 0x7f837006a740 2024-04-25T21:00:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.560+0000 7f836ffff700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f83600a3f70 0x7f83600a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f836ffff700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f83600a3f70 0x7f83600a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8370078e00 tx=0x7f835c00b5f0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f8381a85b80 1 Processor -- start 2024-04-25T21:00:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f8381a85b80 1 -- start start 2024-04-25T21:00:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 0x55bf18ef5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f00d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18f03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f8381a85b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf18e48ec0 con 0x55bf18ef63a0 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f8381a85b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf18defaf0 con 0x55bf18f01270 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.561+0000 7f8381a85b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf18e44e80 con 0x55bf18ef5a80 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f00d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f00d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46248/0 (socket says 172.21.15.47:46248) 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 -- 172.21.15.47:0/3370369384 learned_addr learned my addr 172.21.15.47:0/3370369384 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f8374f98700 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 0x55bf18ef5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f8375799700 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18f03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 -- 172.21.15.47:0/3370369384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 msgr2=0x55bf18ef5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 0x55bf18ef5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 -- 172.21.15.47:0/3370369384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 msgr2=0x55bf18f03660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18f03660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 -- 172.21.15.47:0/3370369384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf18d96a60 con 0x55bf18ef63a0 2024-04-25T21:00:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f8375799700 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18f03660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f836ffff700 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f00d30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f835c00fce0 tx=0x7f835c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa19c9d85dc0ddc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f835affd700 1 -- 172.21.15.47:0/3370369384 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c022070 con 0x55bf18ef63a0 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f835affd700 1 -- 172.21.15.47:0/3370369384 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c01e450 con 0x55bf18ef63a0 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.562+0000 7f835affd700 1 -- 172.21.15.47:0/3370369384 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c01d5b0 con 0x55bf18ef63a0 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 -- 172.21.15.47:0/3370369384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 msgr2=0x55bf18f00d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f00d30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f835c00fce0 tx=0x7f835c012000 comp rx=0 tx=0).stop 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 -- 172.21.15.47:0/3370369384 shutdown_connections 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18f03660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 0x55bf18ef5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 --2- 172.21.15.47:0/3370369384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f00d30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 -- 172.21.15.47:0/3370369384 >> 172.21.15.47:0/3370369384 conn(0x55bf18e23820 msgr2=0x55bf18ef1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 -- 172.21.15.47:0/3370369384 shutdown_connections 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.563+0000 7f8381a85b80 1 -- 172.21.15.47:0/3370369384 wait complete. 2024-04-25T21:00:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 1 -- start start 2024-04-25T21:00:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 1 Processor -- start 2024-04-25T21:00:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 0x55bf18f009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18efdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18efe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf18f03c60 con 0x55bf18ef63a0 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf18de7d60 con 0x55bf18f01270 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf18f03de0 con 0x55bf18ef5a80 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f8381a85b80 10 client.0 _ll_register_callbacks cb 0x55bf18de1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f836ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18efdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.564+0000 7f836ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18efdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46264/0 (socket says 172.21.15.47:46264) 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f836ffff700 1 -- 172.21.15.47:0/2772775866 learned_addr learned my addr 172.21.15.47:0/2772775866 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f8374f98700 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 0x55bf18f009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f8375799700 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18efe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f836ffff700 1 -- 172.21.15.47:0/2772775866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 msgr2=0x55bf18f009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f836ffff700 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 0x55bf18f009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f836ffff700 1 -- 172.21.15.47:0/2772775866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 msgr2=0x55bf18efe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f836ffff700 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18efe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f836ffff700 1 -- 172.21.15.47:0/2772775866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf18e00400 con 0x55bf18ef63a0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f836ffff700 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18efdcc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f835c01ed60 tx=0x7f835c01e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9875395b0e3e8d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f834ffff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f834ffff700 1 -- 172.21.15.47:0/2772775866 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c022070 con 0x55bf18ef63a0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f8381a85b80 1 -- 172.21.15.47:0/2772775866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c01e450 con 0x55bf18ef63a0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f834ffff700 1 -- 172.21.15.47:0/2772775866 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f835c011ae0 con 0x55bf18ef63a0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f834ffff700 1 -- 172.21.15.47:0/2772775866 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f835c01ee60 con 0x55bf18ef63a0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f8381a85b80 10 client.15417 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.565+0000 7f8381a85b80 1 -- 172.21.15.47:0/2772775866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf18f0e4a0 con 0x55bf18ef63a0 2024-04-25T21:00:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f834ffff700 1 -- 172.21.15.47:0/2772775866 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c01e450 con 0x55bf18ef63a0 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f834ffff700 1 -- 172.21.15.47:0/2772775866 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f835c0049e0 con 0x55bf18ef63a0 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f8381a85b80 10 client.15417 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f8381a85b80 10 client.15417 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f8381a85b80 20 client.15417 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bf18f109d0 0x55bf18f12dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f8381a85b80 4 client.15417 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f8381a85b80 1 -- 172.21.15.47:0/2772775866 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf18e566b0 con 0x55bf18f109d0 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.566+0000 7f8374f98700 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bf18f109d0 0x55bf18f12dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.567+0000 7f8374f98700 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bf18f109d0 0x55bf18f12dc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe772e188b0eba6 server_cookie=754e5acf011fed68 in_seq=0 out_seq=0 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.567+0000 7f834ffff700 10 client.15417 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.567+0000 7f834ffff700 1 -- 172.21.15.47:0/2772775866 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf18e566b0 con 0x55bf18f109d0 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.567+0000 7f834ffff700 10 client.15417 handle_command_reply: tid=0 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.567+0000 7f8381a85b80 1 client.15417 shutdown 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 -- 172.21.15.47:0/2772775866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 msgr2=0x55bf18efdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18efdcc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f835c01ed60 tx=0x7f835c01e930 comp rx=0 tx=0).stop 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 -- 172.21.15.47:0/2772775866 shutdown_connections 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bf18f109d0 0x55bf18f12dc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18f01270 0x55bf18efe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef5a80 0x55bf18f009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2772775866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18efdcc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 -- 172.21.15.47:0/2772775866 >> 172.21.15.47:0/2772775866 conn(0x55bf18e23820 msgr2=0x55bf18ef1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f834ffff700 0 client.15417 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 -- 172.21.15.47:0/2772775866 shutdown_connections 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 1 -- 172.21.15.47:0/2772775866 wait complete. 2024-04-25T21:00:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.568+0000 7f8381a85b80 20 client.15417 trim_cache size 0 max 16384 2024-04-25T21:00:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8381a85b80 1 Processor -- start 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8381a85b80 1 -- start start 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18efcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18f01270 0x55bf18f18930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8381a85b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b1cd0 con 0x55bf18ef63a0 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8381a85b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f83701af560 con 0x55bf18ef5a80 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8381a85b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83701ad3f0 con 0x55bf18f01270 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f836ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f836ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f183f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46268/0 (socket says 172.21.15.47:46268) 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f836ffff700 1 -- 172.21.15.47:0/2297614974 learned_addr learned my addr 172.21.15.47:0/2297614974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.589+0000 7f8374f98700 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18efcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f836ffff700 1 -- 172.21.15.47:0/2297614974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18f01270 msgr2=0x55bf18f18930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f836ffff700 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18f01270 0x55bf18f18930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f836ffff700 1 -- 172.21.15.47:0/2297614974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 msgr2=0x55bf18efcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f836ffff700 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18efcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f836ffff700 1 -- 172.21.15.47:0/2297614974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf18ef98c0 con 0x55bf18ef63a0 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f8374f98700 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18efcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f836ffff700 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f183f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f835c004db0 tx=0x7f835c01e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be08196f935e7a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f835a7fc700 1 -- 172.21.15.47:0/2297614974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c015030 con 0x55bf18ef63a0 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f835a7fc700 1 -- 172.21.15.47:0/2297614974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c01eb50 con 0x55bf18ef63a0 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.590+0000 7f835a7fc700 1 -- 172.21.15.47:0/2297614974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c022070 con 0x55bf18ef63a0 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 -- 172.21.15.47:0/2297614974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 msgr2=0x55bf18f183f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f183f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f835c004db0 tx=0x7f835c01e4d0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 -- 172.21.15.47:0/2297614974 shutdown_connections 2024-04-25T21:00:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18f01270 0x55bf18f18930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18efcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 --2- 172.21.15.47:0/2297614974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18ef63a0 0x55bf18f183f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 -- 172.21.15.47:0/2297614974 >> 172.21.15.47:0/2297614974 conn(0x55bf18e23820 msgr2=0x55bf18efc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 -- 172.21.15.47:0/2297614974 shutdown_connections 2024-04-25T21:00:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 -- 172.21.15.47:0/2297614974 wait complete. 2024-04-25T21:00:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 -- start start 2024-04-25T21:00:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.591+0000 7f8381a85b80 1 Processor -- start 2024-04-25T21:00:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18f07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef63a0 0x55bf18f07b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8381a85b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18f01270 0x55bf18f08af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8381a85b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83701ada40 con 0x55bf18f01270 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8381a85b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f83701aee90 con 0x55bf18ef5a80 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8381a85b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83701ae0f0 con 0x55bf18ef63a0 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8381a85b80 10 client.0 _ll_register_callbacks cb 0x55bf18de1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18f01270 0x55bf18f08af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8374f98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18f07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18f01270 0x55bf18f08af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46270/0 (socket says 172.21.15.47:46270) 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8374f98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18f07620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:46482/0 (socket says 172.21.15.47:46482) 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 -- 172.21.15.47:0/296802833 learned_addr learned my addr 172.21.15.47:0/296802833 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f836ffff700 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef63a0 0x55bf18f07b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 -- 172.21.15.47:0/296802833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef63a0 msgr2=0x55bf18f07b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef63a0 0x55bf18f07b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 -- 172.21.15.47:0/296802833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 msgr2=0x55bf18f07620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18f07620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 -- 172.21.15.47:0/296802833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf18f091f0 con 0x55bf18f01270 2024-04-25T21:00:54.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f836ffff700 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef63a0 0x55bf18f07b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f8375799700 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18f01270 0x55bf18f08af0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8368011840 tx=0x7f8368011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50384de08021bd47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.592+0000 7f834ffff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f834ffff700 1 -- 172.21.15.47:0/296802833 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836801b020 con 0x55bf18f01270 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f834ffff700 1 -- 172.21.15.47:0/296802833 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f8368017690 con 0x55bf18f01270 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 1 -- 172.21.15.47:0/296802833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c01eb50 con 0x55bf18f01270 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f834ffff700 1 -- 172.21.15.47:0/296802833 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8368026050 con 0x55bf18f01270 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 10 client.15423 fetch_fsmap learned FSMap version 26 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 1 -- 172.21.15.47:0/296802833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf18f77830 con 0x55bf18f01270 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f834ffff700 1 -- 172.21.15.47:0/296802833 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c01eb50 con 0x55bf18f01270 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f834ffff700 1 -- 172.21.15.47:0/296802833 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f83680159f0 con 0x55bf18f01270 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 10 client.15423 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 10 client.15423 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:00:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 20 client.15423 populate_metadata read hostname 'smithi047' 2024-04-25T21:00:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bf18f77fa0 0x55bf18f7a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 4 client.15423 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}] 2024-04-25T21:00:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.593+0000 7f8381a85b80 1 -- 172.21.15.47:0/296802833 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}) -- 0x55bf18e50750 con 0x55bf18f77fa0 2024-04-25T21:00:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.594+0000 7f8374f98700 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bf18f77fa0 0x55bf18f7a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.594+0000 7f8374f98700 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bf18f77fa0 0x55bf18f7a3c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=716dafb66aa0ead9 server_cookie=28d5561ff8bdf26f in_seq=0 out_seq=0 2024-04-25T21:00:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.594+0000 7f834ffff700 10 client.15423 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.610+0000 7f834ffff700 1 -- 172.21.15.47:0/296802833 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55bf18e50750 con 0x55bf18f77fa0 2024-04-25T21:00:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.610+0000 7f834ffff700 10 client.15423 handle_command_reply: tid=0 2024-04-25T21:00:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.610+0000 7f8381a85b80 1 client.15423 shutdown 2024-04-25T21:00:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 -- 172.21.15.47:0/296802833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18f01270 msgr2=0x55bf18f08af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18f01270 0x55bf18f08af0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8368011840 tx=0x7f8368011b50 comp rx=0 tx=0).stop 2024-04-25T21:00:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 -- 172.21.15.47:0/296802833 shutdown_connections 2024-04-25T21:00:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bf18f77fa0 0x55bf18f7a3c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18f01270 0x55bf18f08af0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf18ef5a80 0x55bf18f07620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 --2- 172.21.15.47:0/296802833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf18ef63a0 0x55bf18f07b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 -- 172.21.15.47:0/296802833 >> 172.21.15.47:0/296802833 conn(0x55bf18e23820 msgr2=0x55bf18efcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f834ffff700 0 client.15423 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 -- 172.21.15.47:0/296802833 shutdown_connections 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 1 -- 172.21.15.47:0/296802833 wait complete. 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.611+0000 7f8381a85b80 20 client.15423 trim_cache size 0 max 16384 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 26, 2024-04-25T21:00:54.613 INFO:teuthology.orchestra.run.smithi047.stdout: "leader": 24488, 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "set_version": 2, 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "sets": { 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "f79dd869": { 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 2, 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "age_ref": 0.0, 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "state": { 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "QUIESCED", 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.0 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "timeout": 60.0, 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "expiration": 86.2, 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "members": { 2024-04-25T21:00:54.614 INFO:teuthology.orchestra.run.smithi047.stdout: "file:/": { 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: "excluded": false, 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: "state": { 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "QUIESCED", 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.0 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:00:54.615 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 -- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f83600a3f70 msgr2=0x7f83600a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f83600a3f70 0x7f83600a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8370078e00 tx=0x7f835c00b5f0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 -- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837006a740 msgr2=0x7f8370198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837006a740 0x7f8370198fc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f83640057c0 tx=0x7f83640057f0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 -- 172.21.15.47:0/1135352331 shutdown_connections 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f83600a3f70 0x7f83600a6430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837006a740 0x7f8370198fc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8370108a60 0x7f837019d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 --2- 172.21.15.47:0/1135352331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837006b100 0x7f8370199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 -- 172.21.15.47:0/1135352331 >> 172.21.15.47:0/1135352331 conn(0x7f83700788e0 msgr2=0x7f83701003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 -- 172.21.15.47:0/1135352331 shutdown_connections 2024-04-25T21:00:54.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:00:54.612+0000 7f83771fc700 1 -- 172.21.15.47:0/1135352331 wait complete. 2024-04-25T21:00:55.414 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f79dd869', quiesce took 7.8 seconds. Will release after: 26.2 2024-04-25T21:00:56.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:56 smithi186 ceph-mon[32529]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T21:00:56.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:56 smithi092 ceph-mon[32524]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T21:00:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:56 smithi047 ceph-mon[29195]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T21:00:59.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:00:58 smithi092 ceph-mon[32524]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:00:59.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:00:58 smithi047 ceph-mon[29195]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:00:59.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:00:58 smithi186 ceph-mon[32529]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:01:01.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:00 smithi092 ceph-mon[32524]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T21:01:01.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:00 smithi047 ceph-mon[29195]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T21:01:01.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:01:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:01:00.904+0000 7f1377f7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:01:01.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:01:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:01:00.904+0000 7f1377f7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:01:01.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:01:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:01:00.904+0000 7f1377f7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:01:01.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:01:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:01:00.904+0000 7f1377f7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:01:01.111 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:01:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:01:00.904+0000 7f1377f7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:01:01.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:00 smithi186 ceph-mon[32529]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T21:01:03.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:02 smithi092 ceph-mon[32524]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T21:01:03.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:02 smithi092 ceph-mon[32524]: mgrmap e24: x(active, since 62s), standbys: z, y 2024-04-25T21:01:03.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:02 smithi047 ceph-mon[29195]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T21:01:03.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:02 smithi047 ceph-mon[29195]: mgrmap e24: x(active, since 62s), standbys: z, y 2024-04-25T21:01:03.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:02 smithi186 ceph-mon[32529]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T21:01:03.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:02 smithi186 ceph-mon[32529]: mgrmap e24: x(active, since 62s), standbys: z, y 2024-04-25T21:01:05.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:04 smithi092 ceph-mon[32524]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T21:01:05.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:04 smithi047 ceph-mon[29195]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T21:01:05.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:04 smithi186 ceph-mon[32529]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T21:01:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:05 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:05 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:05 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:07.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:06 smithi092 ceph-mon[32524]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:01:07.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:06 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:07.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:06 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:07.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:06 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:07.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:06 smithi047 ceph-mon[29195]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:01:07.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:06 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:07.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:06 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:07.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:06 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:07.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:06 smithi186 ceph-mon[32529]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:01:07.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:06 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:07.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:06 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:07.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:06 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:01:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.497+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/4159595593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 msgr2=0x7f0f8410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.497+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/4159595593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 0x7f0f8410bfe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009900 tx=0x7f0f7402a510 comp rx=0 tx=0).stop 2024-04-25T21:01:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.498+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/4159595593 shutdown_connections 2024-04-25T21:01:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.498+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/4159595593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8410c520 0x7f0f8410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.498+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/4159595593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 0x7f0f8410bfe0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.498+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/4159595593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f841031e0 0x7f0f841035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.498+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/4159595593 >> 172.21.15.47:0/4159595593 conn(0x7f0f84077fd0 msgr2=0x7f0f840783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.498+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/4159595593 shutdown_connections 2024-04-25T21:01:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.498+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/4159595593 wait complete. 2024-04-25T21:01:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8c7ad640 1 Processor -- start 2024-04-25T21:01:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8c7ad640 1 -- start start 2024-04-25T21:01:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8c7ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8c7ad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8c7ad640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8c7ad640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84112190 con 0x7f0f84103b00 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8c7ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84112010 con 0x7f0f841031e0 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8c7ad640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84112310 con 0x7f0f8410c520 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f89d48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f89d48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43492/0 (socket says 172.21.15.47:43492) 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f89d48640 1 -- 172.21.15.47:0/2587637750 learned_addr learned my addr 172.21.15.47:0/2587637750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8a549640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f8ad4a640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f89d48640 1 -- 172.21.15.47:0/2587637750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8410c520 msgr2=0x7f0f841018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f89d48640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f89d48640 1 -- 172.21.15.47:0/2587637750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f841031e0 msgr2=0x7f0f84100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f89d48640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.499+0000 7f0f89d48640 1 -- 172.21.15.47:0/2587637750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f841a6bf0 con 0x7f0f84103b00 2024-04-25T21:01:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.500+0000 7f0f89d48640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005940 tx=0x7f0f740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.500+0000 7f0f7b7fe640 1 -- 172.21.15.47:0/2587637750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7403f030 con 0x7f0f84103b00 2024-04-25T21:01:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.500+0000 7f0f7b7fe640 1 -- 172.21.15.47:0/2587637750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f74003cb0 con 0x7f0f84103b00 2024-04-25T21:01:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.500+0000 7f0f7b7fe640 1 -- 172.21.15.47:0/2587637750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74035070 con 0x7f0f84103b00 2024-04-25T21:01:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.501+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/2587637750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f841a6e80 con 0x7f0f84103b00 2024-04-25T21:01:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.501+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/2587637750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841a7230 con 0x7f0f84103b00 2024-04-25T21:01:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.503+0000 7f0f7b7fe640 1 -- 172.21.15.47:0/2587637750 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f0f74042030 con 0x7f0f84103b00 2024-04-25T21:01:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.504+0000 7f0f7b7fe640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f700a3f70 0x7f0f700a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.504+0000 7f0f7b7fe640 1 -- 172.21.15.47:0/2587637750 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f0f740f1280 con 0x7f0f84103b00 2024-04-25T21:01:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.504+0000 7f0f8a549640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f700a3f70 0x7f0f700a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.504+0000 7f0f8f5a2740 1 Processor -- start 2024-04-25T21:01:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8f5a2740 1 -- start start 2024-04-25T21:01:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8a549640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f700a3f70 0x7f0f700a6430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000bbe0 tx=0x7f0f8000b390 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff63a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff63f40 0x55f34ff6e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff6e800 0x55f34ff70bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8f5a2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f34fec8970 con 0x55f34ff63620 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8f5a2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f34fe654a0 con 0x55f34ff63f40 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8f5a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f34fec4930 con 0x55f34ff6e800 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8a549640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff63a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8a549640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff63a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43508/0 (socket says 172.21.15.47:43508) 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8a549640 1 -- 172.21.15.47:0/1303630959 learned_addr learned my addr 172.21.15.47:0/1303630959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f89d48640 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff63f40 0x55f34ff6e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8ad4a640 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff6e800 0x55f34ff70bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f89d48640 1 -- 172.21.15.47:0/1303630959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff6e800 msgr2=0x55f34ff70bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f89d48640 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff6e800 0x55f34ff70bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f89d48640 1 -- 172.21.15.47:0/1303630959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 msgr2=0x55f34ff63a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f89d48640 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff63a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f89d48640 1 -- 172.21.15.47:0/1303630959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34fbdb210 con 0x55f34ff63f40 2024-04-25T21:01:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.505+0000 7f0f8a549640 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff63a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.506+0000 7f0f89d48640 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff63f40 0x55f34ff6e2c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0f740ee7e0 tx=0x7f0f74032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81349720c28ba0f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.506+0000 7f0f78ff9640 1 -- 172.21.15.47:0/1303630959 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740bd730 con 0x55f34ff63f40 2024-04-25T21:01:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.506+0000 7f0f78ff9640 1 -- 172.21.15.47:0/1303630959 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f740bd030 con 0x55f34ff63f40 2024-04-25T21:01:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.506+0000 7f0f78ff9640 1 -- 172.21.15.47:0/1303630959 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740bd290 con 0x55f34ff63f40 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1303630959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff63f40 msgr2=0x55f34ff6e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff63f40 0x55f34ff6e2c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0f740ee7e0 tx=0x7f0f74032f60 comp rx=0 tx=0).stop 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1303630959 shutdown_connections 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff6e800 0x55f34ff70bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff63f40 0x55f34ff6e2c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/1303630959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff63a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1303630959 >> 172.21.15.47:0/1303630959 conn(0x55f34ff5fb30 msgr2=0x55f34ff5ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1303630959 shutdown_connections 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.507+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1303630959 wait complete. 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f8f5a2740 1 -- start start 2024-04-25T21:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f8f5a2740 1 Processor -- start 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff6e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63f40 0x55f34ff6b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f8a549640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff6e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f8a549640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff6e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43518/0 (socket says 172.21.15.47:43518) 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f8a549640 1 -- 172.21.15.47:0/569047134 learned_addr learned my addr 172.21.15.47:0/569047134 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff6b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.508+0000 7f0f89d48640 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63f40 0x55f34ff6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f34fe654a0 con 0x55f34ff63620 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f34fe630b0 con 0x55f34ff6e800 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f34ff711f0 con 0x55f34ff63f40 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8ad4a640 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff6b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8a549640 1 -- 172.21.15.47:0/569047134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63f40 msgr2=0x55f34ff6b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8a549640 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63f40 0x55f34ff6b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8a549640 1 -- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 msgr2=0x55f34ff6b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8a549640 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff6b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8a549640 1 -- 172.21.15.47:0/569047134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34fca7620 con 0x55f34ff63620 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f89d48640 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63f40 0x55f34ff6b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8a549640 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff6e110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0f800117c0 tx=0x7f0f80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d6ec82c12926f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8f5a2740 10 client.0 _ll_register_callbacks cb 0x55f34fe50d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f6dffb640 1 -- 172.21.15.47:0/569047134 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8001fbc0 con 0x55f34ff63620 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.509+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f740bd030 con 0x55f34ff63620 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f6dffb640 1 -- 172.21.15.47:0/569047134 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f0f8001fd60 con 0x55f34ff63620 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f6dffb640 1 -- 172.21.15.47:0/569047134 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f80028440 con 0x55f34ff63620 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f8f5a2740 10 client.15441 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55f34ff7cda0 con 0x55f34ff63620 2024-04-25T21:01:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f6dffb640 1 -- 172.21.15.47:0/569047134 <== mon.0 v2:172.21.15.47:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0f80026070 con 0x55f34ff63620 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f8f5a2740 10 client.15441 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f8f5a2740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f8f5a2740 20 client.15441 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f34ff7e150 0x55f34ff805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f8f5a2740 4 client.15441 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.510+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0f740bd030 con 0x55f34ff7e150 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.511+0000 7f0f89d48640 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f34ff7e150 0x55f34ff805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.511+0000 7f0f6dffb640 10 client.15441 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.511+0000 7f0f89d48640 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f34ff7e150 0x55f34ff805a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7799bea43f3e145 server_cookie=427b4253ca93316d in_seq=0 out_seq=0 2024-04-25T21:01:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f6dffb640 1 -- 172.21.15.47:0/569047134 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f0f7409be50 con 0x55f34ff7e150 2024-04-25T21:01:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f6dffb640 10 client.15441 handle_command_reply: tid=0 2024-04-25T21:01:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 client.15441 shutdown 2024-04-25T21:01:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 msgr2=0x55f34ff6e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff6e110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0f800117c0 tx=0x7f0f80004730 comp rx=0 tx=0).stop 2024-04-25T21:01:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 shutdown_connections 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f34ff7e150 0x55f34ff805a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f6dffb640 0 client.15441 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff6b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63f40 0x55f34ff6b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/569047134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63620 0x55f34ff6e110 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 >> 172.21.15.47:0/569047134 conn(0x55f34ff5fb30 msgr2=0x55f34ff658b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 shutdown_connections 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/569047134 wait complete. 2024-04-25T21:01:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.512+0000 7f0f8f5a2740 20 client.15441 trim_cache size 0 max 16384 2024-04-25T21:01:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.527+0000 7f0f8f5a2740 1 Processor -- start 2024-04-25T21:01:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.527+0000 7f0f8f5a2740 1 -- start start 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 0x55f34ff79ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63f40 0x55f34ff6b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff5e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8f5a2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841b25b0 con 0x55f34ff63f40 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8f5a2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841afe80 con 0x55f34ff6e800 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8f5a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841adcf0 con 0x55f34ff63620 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f89d48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63f40 0x55f34ff6b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f89d48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63f40 0x55f34ff6b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43524/0 (socket says 172.21.15.47:43524) 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f89d48640 1 -- 172.21.15.47:0/1591270342 learned_addr learned my addr 172.21.15.47:0/1591270342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8ad4a640 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff5e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8a549640 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 0x55f34ff79ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f89d48640 1 -- 172.21.15.47:0/1591270342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 msgr2=0x55f34ff79ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f89d48640 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 0x55f34ff79ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f89d48640 1 -- 172.21.15.47:0/1591270342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 msgr2=0x55f34ff5e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f89d48640 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff5e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f89d48640 1 -- 172.21.15.47:0/1591270342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34ff69950 con 0x55f34ff63f40 2024-04-25T21:01:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.528+0000 7f0f8ad4a640 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff5e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.529+0000 7f0f89d48640 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63f40 0x55f34ff6b8e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0f74006010 tx=0x7f0f740bcca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b313d245e8575f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.529+0000 7f0f6ffff640 1 -- 172.21.15.47:0/1591270342 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740c10b0 con 0x55f34ff63f40 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.529+0000 7f0f6ffff640 1 -- 172.21.15.47:0/1591270342 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f740bd030 con 0x55f34ff63f40 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.529+0000 7f0f6ffff640 1 -- 172.21.15.47:0/1591270342 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740c5810 con 0x55f34ff63f40 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.529+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1591270342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63f40 msgr2=0x55f34ff6b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.529+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63f40 0x55f34ff6b8e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0f74006010 tx=0x7f0f740bcca0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1591270342 shutdown_connections 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff6e800 0x55f34ff5e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff63f40 0x55f34ff6b8e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/1591270342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 0x55f34ff79ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1591270342 >> 172.21.15.47:0/1591270342 conn(0x55f34ff5fb30 msgr2=0x55f34ff6b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1591270342 shutdown_connections 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/1591270342 wait complete. 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 -- start start 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 Processor -- start 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 0x55f34ff74000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff6e800 0x55f34ff74540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f8f5a2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff7e150 0x55f34ff74a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.530+0000 7f0f89d48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff6e800 0x55f34ff74540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f8f5a2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ae340 con 0x55f34ff6e800 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f89d48640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff6e800 0x55f34ff74540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43534/0 (socket says 172.21.15.47:43534) 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f89d48640 1 -- 172.21.15.47:0/2667244687 learned_addr learned my addr 172.21.15.47:0/2667244687 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841af7b0 con 0x55f34ff7e150 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841aea10 con 0x55f34ff63620 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f8f5a2740 10 client.0 _ll_register_callbacks cb 0x7f0f80011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f8a549640 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 0x55f34ff74000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f8ad4a640 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff7e150 0x55f34ff74a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f89d48640 1 -- 172.21.15.47:0/2667244687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 msgr2=0x55f34ff74000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f89d48640 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 0x55f34ff74000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f89d48640 1 -- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff7e150 msgr2=0x55f34ff74a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f89d48640 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff7e150 0x55f34ff74a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f89d48640 1 -- 172.21.15.47:0/2667244687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34ffe3980 con 0x55f34ff6e800 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f8ad4a640 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff7e150 0x55f34ff74a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f89d48640 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff6e800 0x55f34ff74540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0f740bcfc0 tx=0x7f0f7403b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5813ace9875728f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.531+0000 7f0f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f6dffb640 1 -- 172.21.15.47:0/2667244687 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740bd390 con 0x55f34ff6e800 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f6dffb640 1 -- 172.21.15.47:0/2667244687 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f0f740c1040 con 0x55f34ff6e800 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f740bd030 con 0x55f34ff6e800 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f6dffb640 1 -- 172.21.15.47:0/2667244687 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f740c5810 con 0x55f34ff6e800 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f8f5a2740 10 client.15447 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f34ffe3980 con 0x55f34ff6e800 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f6dffb640 1 -- 172.21.15.47:0/2667244687 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7409be50 con 0x55f34ff6e800 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f6dffb640 1 -- 172.21.15.47:0/2667244687 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0f7404c060 con 0x55f34ff6e800 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f8f5a2740 10 client.15447 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f8f5a2740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f8f5a2740 20 client.15447 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.532+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f34ffe4e90 0x55f34ffe7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.533+0000 7f0f8f5a2740 4 client.15447 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.533+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0f740bd030 con 0x55f34ffe4e90 2024-04-25T21:01:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.533+0000 7f0f8a549640 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f34ffe4e90 0x55f34ffe7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.534+0000 7f0f6dffb640 10 client.15447 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.534+0000 7f0f8a549640 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f34ffe4e90 0x55f34ffe7280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d39a099efdaefa server_cookie=5867043644f07c88 in_seq=0 out_seq=0 2024-04-25T21:01:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f6dffb640 1 -- 172.21.15.47:0/2667244687 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0f740bd030 con 0x55f34ffe4e90 2024-04-25T21:01:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f6dffb640 10 client.15447 handle_command_reply: tid=0 2024-04-25T21:01:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 client.15447 shutdown 2024-04-25T21:01:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff6e800 msgr2=0x55f34ff74540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff6e800 0x55f34ff74540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0f740bcfc0 tx=0x7f0f7403b6b0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 shutdown_connections 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f34ffe4e90 0x55f34ffe7280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34ff7e150 0x55f34ff74a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f6dffb640 0 client.15447 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f34ff6e800 0x55f34ff74540 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 --2- 172.21.15.47:0/2667244687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f34ff63620 0x55f34ff74000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 >> 172.21.15.47:0/2667244687 conn(0x55f34ff5fb30 msgr2=0x55f34ff78e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 shutdown_connections 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.541+0000 7f0f8f5a2740 1 -- 172.21.15.47:0/2667244687 wait complete. 2024-04-25T21:01:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8f5a2740 20 client.15447 trim_cache size 0 max 16384 2024-04-25T21:01:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:01:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:01:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:01:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f700a3f70 msgr2=0x7f0f700a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f700a3f70 0x7f0f700a6430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000bbe0 tx=0x7f0f8000b390 comp rx=0 tx=0).stop 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 msgr2=0x7f0f84101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005940 tx=0x7f0f740042f0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/2587637750 shutdown_connections 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f700a3f70 0x7f0f700a6430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 --2- 172.21.15.47:0/2587637750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/2587637750 >> 172.21.15.47:0/2587637750 conn(0x7f0f84077fd0 msgr2=0x7f0f840fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/2587637750 shutdown_connections 2024-04-25T21:01:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:07.542+0000 7f0f8c7ad640 1 -- 172.21.15.47:0/2587637750 wait complete. 2024-04-25T21:01:08.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:07 smithi047 ceph-mon[29195]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:01:08.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:07 smithi092 ceph-mon[32524]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:01:08.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:07 smithi186 ceph-mon[32529]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:01:10.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:10 smithi047 ceph-mon[29195]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:01:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:10 smithi186 ceph-mon[32529]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:01:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:10 smithi092 ceph-mon[32524]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:01:12.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:12 smithi047 ceph-mon[29195]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T21:01:12.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:12 smithi186 ceph-mon[32529]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T21:01:12.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:12 smithi092 ceph-mon[32524]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T21:01:14.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:14 smithi047 ceph-mon[29195]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T21:01:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:14 smithi186 ceph-mon[32529]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T21:01:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:14 smithi092 ceph-mon[32524]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T21:01:16.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:16 smithi047 ceph-mon[29195]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T21:01:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:16 smithi186 ceph-mon[32529]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T21:01:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:16 smithi092 ceph-mon[32524]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T21:01:18.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:18 smithi047 ceph-mon[29195]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T21:01:18.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:18 smithi186 ceph-mon[32529]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T21:01:18.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:18 smithi092 ceph-mon[32524]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T21:01:20.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:20 smithi047 ceph-mon[29195]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T21:01:20.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:20 smithi186 ceph-mon[32529]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T21:01:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:20 smithi092 ceph-mon[32524]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T21:01:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.684+0000 7f581fed0640 1 -- 172.21.15.47:0/861101673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f581810c560 msgr2=0x7f581810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.684+0000 7f581fed0640 1 --2- 172.21.15.47:0/861101673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f581810c560 0x7f581810e9b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5810009ca0 tx=0x7f581002a950 comp rx=0 tx=0).stop 2024-04-25T21:01:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.685+0000 7f581fed0640 1 -- 172.21.15.47:0/861101673 shutdown_connections 2024-04-25T21:01:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.685+0000 7f581fed0640 1 --2- 172.21.15.47:0/861101673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f581810c560 0x7f581810e9b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.685+0000 7f581fed0640 1 --2- 172.21.15.47:0/861101673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5818103b40 0x7f581810c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.685+0000 7f581fed0640 1 --2- 172.21.15.47:0/861101673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5818103220 0x7f5818103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.685+0000 7f581fed0640 1 -- 172.21.15.47:0/861101673 >> 172.21.15.47:0/861101673 conn(0x7f5818077fd0 msgr2=0x7f58180783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.685+0000 7f581fed0640 1 -- 172.21.15.47:0/861101673 shutdown_connections 2024-04-25T21:01:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.685+0000 7f581fed0640 1 -- 172.21.15.47:0/861101673 wait complete. 2024-04-25T21:01:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581fed0640 1 Processor -- start 2024-04-25T21:01:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581fed0640 1 -- start start 2024-04-25T21:01:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581fed0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5818103220 0x7f58181985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581fed0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5818103b40 0x7f5818198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581fed0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f581810c560 0x7f581819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581fed0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f58181121a0 con 0x7f5818103b40 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581fed0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5818112020 con 0x7f581810c560 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581fed0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5818112320 con 0x7f5818103220 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581dc6c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5818103220 0x7f58181985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581dc6c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5818103220 0x7f58181985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:48660/0 (socket says 172.21.15.47:48660) 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581dc6c640 1 -- 172.21.15.47:0/1151513400 learned_addr learned my addr 172.21.15.47:0/1151513400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.686+0000 7f581e46d640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f581810c560 0x7f581819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581d46b640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5818103b40 0x7f5818198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581dc6c640 1 -- 172.21.15.47:0/1151513400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f581810c560 msgr2=0x7f581819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581dc6c640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f581810c560 0x7f581819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581dc6c640 1 -- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5818103b40 msgr2=0x7f5818198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581dc6c640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5818103b40 0x7f5818198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581dc6c640 1 -- 172.21.15.47:0/1151513400 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581819d5b0 con 0x7f5818103220 2024-04-25T21:01:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581d46b640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5818103b40 0x7f5818198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581dc6c640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5818103220 0x7f58181985c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f581400d770 tx=0x7f581400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f580effd640 1 -- 172.21.15.47:0/1151513400 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814017070 con 0x7f5818103220 2024-04-25T21:01:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f581fed0640 1 -- 172.21.15.47:0/1151513400 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581819d8a0 con 0x7f5818103220 2024-04-25T21:01:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f580effd640 1 -- 172.21.15.47:0/1151513400 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58140054d0 con 0x7f5818103220 2024-04-25T21:01:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.687+0000 7f580effd640 1 -- 172.21.15.47:0/1151513400 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814004c30 con 0x7f5818103220 2024-04-25T21:01:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.688+0000 7f581fed0640 1 -- 172.21.15.47:0/1151513400 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a5130 con 0x7f5818103220 2024-04-25T21:01:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f580effd640 1 -- 172.21.15.47:0/1151513400 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f5814004e10 con 0x7f5818103220 2024-04-25T21:01:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f5822be1740 1 Processor -- start 2024-04-25T21:01:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f5822be1740 1 -- start start 2024-04-25T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f5822be1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca4ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f5822be1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f5822be1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca5bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f5822be1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55628c9b3970 con 0x55628ca4e620 2024-04-25T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f5822be1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55628c9504a0 con 0x55628ca59800 2024-04-25T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.691+0000 7f5822be1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55628c9af930 con 0x55628ca4ef40 2024-04-25T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581dc6c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581e46d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca4ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581dc6c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:48668/0 (socket says 172.21.15.47:48668) 2024-04-25T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581d46b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca5bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581dc6c640 1 -- 172.21.15.47:0/3469248753 learned_addr learned my addr 172.21.15.47:0/3469248753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f580effd640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f58040a3ff0 0x7f58040a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581d46b640 1 -- 172.21.15.47:0/3469248753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 msgr2=0x55628ca592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581d46b640 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581d46b640 1 -- 172.21.15.47:0/3469248753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 msgr2=0x55628ca4ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f580effd640 1 -- 172.21.15.47:0/1151513400 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f58140d0520 con 0x7f5818103220 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581d46b640 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca4ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581d46b640 1 -- 172.21.15.47:0/3469248753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55628c6c6210 con 0x55628ca59800 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581dc6c640 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.692+0000 7f581e46d640 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca4ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.693+0000 7f581d46b640 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca5bbf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5808008b00 tx=0x7f580800e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88aeccc2f588739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.693+0000 7f581e46d640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f58040a3ff0 0x7f58040a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.693+0000 7f5803fff640 1 -- 172.21.15.47:0/3469248753 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808014070 con 0x55628ca59800 2024-04-25T21:01:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.693+0000 7f5803fff640 1 -- 172.21.15.47:0/3469248753 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58080054d0 con 0x55628ca59800 2024-04-25T21:01:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.693+0000 7f5803fff640 1 -- 172.21.15.47:0/3469248753 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808014070 con 0x55628ca59800 2024-04-25T21:01:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 -- 172.21.15.47:0/3469248753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 msgr2=0x55628ca5bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f581e46d640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f58040a3ff0 0x7f58040a64b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5810005fe0 tx=0x7f581003c040 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca5bbf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5808008b00 tx=0x7f580800e350 comp rx=0 tx=0).stop 2024-04-25T21:01:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 -- 172.21.15.47:0/3469248753 shutdown_connections 2024-04-25T21:01:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca5bbf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 --2- 172.21.15.47:0/3469248753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca4ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 -- 172.21.15.47:0/3469248753 >> 172.21.15.47:0/3469248753 conn(0x55628ca4ab30 msgr2=0x55628ca4af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 -- 172.21.15.47:0/3469248753 shutdown_connections 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 -- 172.21.15.47:0/3469248753 wait complete. 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.694+0000 7f5822be1740 1 -- start start 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f5822be1740 1 Processor -- start 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f5822be1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f5822be1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f5822be1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca56900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f5822be1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55628ca5c430 con 0x55628ca4e620 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f5822be1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55628c94e0b0 con 0x55628ca59800 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f5822be1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55628ca5c5b0 con 0x55628ca4ef40 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f5822be1740 10 client.0 _ll_register_callbacks cb 0x55628c93bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f581dc6c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f581dc6c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38084/0 (socket says 172.21.15.47:38084) 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f581dc6c640 1 -- 172.21.15.47:0/3737565733 learned_addr learned my addr 172.21.15.47:0/3737565733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.695+0000 7f581e46d640 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca56900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f581d46b640 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f581dc6c640 1 -- 172.21.15.47:0/3737565733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 msgr2=0x55628ca563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f581dc6c640 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f581dc6c640 1 -- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 msgr2=0x55628ca56900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f581dc6c640 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca56900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f581dc6c640 1 -- 172.21.15.47:0/3737565733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55628c792620 con 0x55628ca4e620 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f581e46d640 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca56900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f581dc6c640 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca590d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f58140cdb30 tx=0x7f581409bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdde16b6080f3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f58017fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f58017fa640 1 -- 172.21.15.47:0/3737565733 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814020040 con 0x55628ca4e620 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.696+0000 7f58017fa640 1 -- 172.21.15.47:0/3737565733 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f58140a4a70 con 0x55628ca4e620 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.697+0000 7f5822be1740 1 -- 172.21.15.47:0/3737565733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58080054d0 con 0x55628ca4e620 2024-04-25T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.697+0000 7f58017fa640 1 -- 172.21.15.47:0/3737565733 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f581409c990 con 0x55628ca4e620 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.697+0000 7f5822be1740 10 client.15459 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.697+0000 7f5822be1740 1 -- 172.21.15.47:0/3737565733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55628ca6c750 con 0x55628ca4e620 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.697+0000 7f58017fa640 1 -- 172.21.15.47:0/3737565733 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58080054d0 con 0x55628ca4e620 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.697+0000 7f58017fa640 1 -- 172.21.15.47:0/3737565733 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f58140a0120 con 0x55628ca4e620 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.697+0000 7f5822be1740 10 client.15459 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.698+0000 7f5822be1740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.698+0000 7f5822be1740 20 client.15459 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.698+0000 7f5822be1740 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55628ca6dbc0 0x55628ca6ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.698+0000 7f5822be1740 4 client.15459 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.698+0000 7f5822be1740 1 -- 172.21.15.47:0/3737565733 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55628c9c1390 con 0x55628ca6dbc0 2024-04-25T21:01:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.698+0000 7f581d46b640 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55628ca6dbc0 0x55628ca6ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.699+0000 7f58017fa640 10 client.15459 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.699+0000 7f581d46b640 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55628ca6dbc0 0x55628ca6ffb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4792a5ebbb4cc896 server_cookie=dd5827b097ce7fb3 in_seq=0 out_seq=0 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.699+0000 7f58017fa640 1 -- 172.21.15.47:0/3737565733 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55628c9c1390 con 0x55628ca6dbc0 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.699+0000 7f58017fa640 10 client.15459 handle_command_reply: tid=0 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.699+0000 7f5822be1740 1 client.15459 shutdown 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 -- 172.21.15.47:0/3737565733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 msgr2=0x55628ca590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca590d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f58140cdb30 tx=0x7f581409bd20 comp rx=0 tx=0).stop 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 -- 172.21.15.47:0/3737565733 shutdown_connections 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55628ca6dbc0 0x55628ca6ffb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca59800 0x55628ca56900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 --2- 172.21.15.47:0/3737565733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca590d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 -- 172.21.15.47:0/3737565733 >> 172.21.15.47:0/3737565733 conn(0x55628ca4ab30 msgr2=0x55628ca508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f58017fa640 0 client.15459 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 -- 172.21.15.47:0/3737565733 shutdown_connections 2024-04-25T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 1 -- 172.21.15.47:0/3737565733 wait complete. 2024-04-25T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.700+0000 7f5822be1740 20 client.15459 trim_cache size 0 max 16384 2024-04-25T21:01:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f5822be1740 1 Processor -- start 2024-04-25T21:01:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f5822be1740 1 -- start start 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f5822be1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ab30 0x55628ca652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f5822be1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca51970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f5822be1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ef40 0x55628ca58c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f581d46b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca51970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f5822be1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f58181ae190 con 0x55628ca4e620 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f5822be1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f58181aba60 con 0x55628ca4ef40 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f5822be1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f58181a98d0 con 0x55628ca4ab30 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f581d46b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca51970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38092/0 (socket says 172.21.15.47:38092) 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f581d46b640 1 -- 172.21.15.47:0/2952679635 learned_addr learned my addr 172.21.15.47:0/2952679635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f581dc6c640 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ab30 0x55628ca652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.730+0000 7f581e46d640 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ef40 0x55628ca58c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f581dc6c640 1 -- 172.21.15.47:0/2952679635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ef40 msgr2=0x55628ca58c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f581dc6c640 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ef40 0x55628ca58c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f581dc6c640 1 -- 172.21.15.47:0/2952679635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 msgr2=0x55628ca51970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f581dc6c640 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca51970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f581dc6c640 1 -- 172.21.15.47:0/2952679635 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55628ca54910 con 0x55628ca4ab30 2024-04-25T21:01:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f581d46b640 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca51970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f581dc6c640 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ab30 0x55628ca652f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f581409bf90 tx=0x7f5814005af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dfdf0437d5b549c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f58037fe640 1 -- 172.21.15.47:0/2952679635 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814015040 con 0x55628ca4ab30 2024-04-25T21:01:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f58037fe640 1 -- 172.21.15.47:0/2952679635 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58140a0640 con 0x55628ca4ab30 2024-04-25T21:01:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.731+0000 7f58037fe640 1 -- 172.21.15.47:0/2952679635 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814020040 con 0x55628ca4ab30 2024-04-25T21:01:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 -- 172.21.15.47:0/2952679635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ab30 msgr2=0x55628ca652f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ab30 0x55628ca652f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f581409bf90 tx=0x7f5814005af0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 -- 172.21.15.47:0/2952679635 shutdown_connections 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ef40 0x55628ca58c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca51970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 --2- 172.21.15.47:0/2952679635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ab30 0x55628ca652f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 -- 172.21.15.47:0/2952679635 >> 172.21.15.47:0/2952679635 conn(0x55628ca56a60 msgr2=0x55628ca51260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 -- 172.21.15.47:0/2952679635 shutdown_connections 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.732+0000 7f5822be1740 1 -- 172.21.15.47:0/2952679635 wait complete. 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 1 -- start start 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 1 Processor -- start 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ab30 0x55628ca61750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca5cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f58181a9f20 con 0x55628ca4e620 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f58181ab390 con 0x55628ca4ab30 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f58181aa5f0 con 0x55628ca4ef40 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f5822be1740 10 client.0 _ll_register_callbacks cb 0x7f581400de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f581d46b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f581d46b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca61c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38102/0 (socket says 172.21.15.47:38102) 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f581d46b640 1 -- 172.21.15.47:0/927020295 learned_addr learned my addr 172.21.15.47:0/927020295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.733+0000 7f581dc6c640 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ab30 0x55628ca61750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f581e46d640 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca5cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f581d46b640 1 -- 172.21.15.47:0/927020295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 msgr2=0x55628ca5cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f581d46b640 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca5cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f581d46b640 1 -- 172.21.15.47:0/927020295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ab30 msgr2=0x55628ca61750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f581d46b640 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ab30 0x55628ca61750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f581d46b640 1 -- 172.21.15.47:0/927020295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55628ca5d4a0 con 0x55628ca4e620 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f581d46b640 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca61c90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55628ca4bcc0 tx=0x7f5808008450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e709f586e32e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f58017fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f58017fa640 1 -- 172.21.15.47:0/927020295 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808014070 con 0x55628ca4e620 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f58017fa640 1 -- 172.21.15.47:0/927020295 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f5808004250 con 0x55628ca4e620 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.734+0000 7f5822be1740 1 -- 172.21.15.47:0/927020295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58140a0640 con 0x55628ca4e620 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f58017fa640 1 -- 172.21.15.47:0/927020295 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5808036050 con 0x55628ca4e620 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f5822be1740 10 client.15465 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f5822be1740 1 -- 172.21.15.47:0/927020295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55628cad3c80 con 0x55628ca4e620 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f58017fa640 1 -- 172.21.15.47:0/927020295 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58140a0640 con 0x55628ca4e620 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f58017fa640 1 -- 172.21.15.47:0/927020295 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5808003710 con 0x55628ca4e620 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f5822be1740 10 client.15465 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f5822be1740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f5822be1740 20 client.15465 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f5822be1740 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55628ca59800 0x55628cad6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f5822be1740 4 client.15465 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f5822be1740 1 -- 172.21.15.47:0/927020295 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55628c9bb200 con 0x55628ca59800 2024-04-25T21:01:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.735+0000 7f581dc6c640 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55628ca59800 0x55628cad6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.736+0000 7f58017fa640 10 client.15465 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.736+0000 7f581dc6c640 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55628ca59800 0x55628cad6210 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f10e603c096a4ebe server_cookie=24c1adfd3c056b22 in_seq=0 out_seq=0 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f58017fa640 1 -- 172.21.15.47:0/927020295 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55628c9bb200 con 0x55628ca59800 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f58017fa640 10 client.15465 handle_command_reply: tid=0 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 client.15465 shutdown 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 -- 172.21.15.47:0/927020295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 msgr2=0x55628ca61c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca61c90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55628ca4bcc0 tx=0x7f5808008450 comp rx=0 tx=0).stop 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 -- 172.21.15.47:0/927020295 shutdown_connections 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55628ca59800 0x55628cad6210 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55628ca4ef40 0x55628ca5cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55628ca4e620 0x55628ca61c90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 --2- 172.21.15.47:0/927020295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55628ca4ab30 0x55628ca61750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 -- 172.21.15.47:0/927020295 >> 172.21.15.47:0/927020295 conn(0x55628ca56a60 msgr2=0x55628ca49cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f58017fa640 0 client.15465 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 -- 172.21.15.47:0/927020295 shutdown_connections 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 1 -- 172.21.15.47:0/927020295 wait complete. 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.740+0000 7f5822be1740 20 client.15465 trim_cache size 0 max 16384 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 -- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f58040a3ff0 msgr2=0x7f58040a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f58040a3ff0 0x7f58040a64b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5810005fe0 tx=0x7f581003c040 comp rx=0 tx=0).stop 2024-04-25T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 -- 172.21.15.47:0/1151513400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5818103220 msgr2=0x7f58181985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5818103220 0x7f58181985c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f581400d770 tx=0x7f581400dc30 comp rx=0 tx=0).stop 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 -- 172.21.15.47:0/1151513400 shutdown_connections 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f58040a3ff0 0x7f58040a64b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f581810c560 0x7f581819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5818103b40 0x7f5818198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 --2- 172.21.15.47:0/1151513400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5818103220 0x7f58181985c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 -- 172.21.15.47:0/1151513400 >> 172.21.15.47:0/1151513400 conn(0x7f5818077fd0 msgr2=0x7f58180fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 -- 172.21.15.47:0/1151513400 shutdown_connections 2024-04-25T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:20.741+0000 7f581fed0640 1 -- 172.21.15.47:0/1151513400 wait complete. 2024-04-25T21:01:21.615 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:01:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:01:22.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:22 smithi047 ceph-mon[29195]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T21:01:22.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:22 smithi186 ceph-mon[32529]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T21:01:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:22 smithi092 ceph-mon[32524]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T21:01:24.355 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:24 smithi047 ceph-mon[29195]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:01:24.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:24 smithi186 ceph-mon[32529]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:01:24.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:24 smithi092 ceph-mon[32524]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:01:25.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.694+0000 7f2bb7182700 1 -- 172.21.15.47:0/3222844685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101360 msgr2=0x7f2bb0101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.694+0000 7f2bb7182700 1 --2- 172.21.15.47:0/3222844685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101360 0x7f2bb0101740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40099c0 tx=0x7f2ba4009cd0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.694+0000 7f2bb7182700 1 -- 172.21.15.47:0/3222844685 shutdown_connections 2024-04-25T21:01:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.694+0000 7f2bb7182700 1 --2- 172.21.15.47:0/3222844685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101360 0x7f2bb0101740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.695+0000 7f2bb7182700 1 --2- 172.21.15.47:0/3222844685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb01063f0 0x7f2bb010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.695+0000 7f2bb7182700 1 --2- 172.21.15.47:0/3222844685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb0101d10 0x7f2bb0105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.695+0000 7f2bb7182700 1 -- 172.21.15.47:0/3222844685 >> 172.21.15.47:0/3222844685 conn(0x7f2bb0077fb0 msgr2=0x7f2bb00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.695+0000 7f2bb7182700 1 -- 172.21.15.47:0/3222844685 shutdown_connections 2024-04-25T21:01:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.695+0000 7f2bb7182700 1 -- 172.21.15.47:0/3222844685 wait complete. 2024-04-25T21:01:25.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb7182700 1 Processor -- start 2024-04-25T21:01:25.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb7182700 1 -- start start 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb7182700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb0101360 0x7f2bb00fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb7182700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101d10 0x7f2bb00fd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb7182700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb01063f0 0x7f2bb00fdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2baffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101d10 0x7f2bb00fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2baffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101d10 0x7f2bb00fd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40272/0 (socket says 172.21.15.47:40272) 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2baffff700 1 -- 172.21.15.47:0/558121429 learned_addr learned my addr 172.21.15.47:0/558121429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0110a70 con 0x7f2bb0101d10 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01108f0 con 0x7f2bb01063f0 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0110bf0 con 0x7f2bb0101360 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.696+0000 7f2bb4f1e700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb0101360 0x7f2bb00fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2bb571f700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb01063f0 0x7f2bb00fdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2baffff700 1 -- 172.21.15.47:0/558121429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb0101360 msgr2=0x7f2bb00fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:25.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2baffff700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb0101360 0x7f2bb00fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2baffff700 1 -- 172.21.15.47:0/558121429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb01063f0 msgr2=0x7f2bb00fdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:25.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2baffff700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb01063f0 0x7f2bb00fdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2baffff700 1 -- 172.21.15.47:0/558121429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb01abaf0 con 0x7f2bb0101d10 2024-04-25T21:01:25.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2baffff700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101d10 0x7f2bb00fd960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00d950 tx=0x7f2b9c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:25.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2badffb700 1 -- 172.21.15.47:0/558121429 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c016070 con 0x7f2bb0101d10 2024-04-25T21:01:25.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb01abd80 con 0x7f2bb0101d10 2024-04-25T21:01:25.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.698+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb01ac1b0 con 0x7f2bb0101d10 2024-04-25T21:01:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.699+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb01027f0 con 0x7f2bb0101d10 2024-04-25T21:01:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.697+0000 7f2badffb700 1 -- 172.21.15.47:0/558121429 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c004830 con 0x7f2bb0101d10 2024-04-25T21:01:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.706+0000 7f2badffb700 1 -- 172.21.15.47:0/558121429 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0054c0 con 0x7f2bb0101d10 2024-04-25T21:01:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.706+0000 7f2badffb700 1 -- 172.21.15.47:0/558121429 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f2b9c00b750 con 0x7f2bb0101d10 2024-04-25T21:01:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.707+0000 7f2badffb700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ba00a4160 0x7f2ba00a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.707+0000 7f2badffb700 1 -- 172.21.15.47:0/558121429 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f2b9c0d1d10 con 0x7f2bb0101d10 2024-04-25T21:01:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.707+0000 7f2bb4f1e700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ba00a4160 0x7f2ba00a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.707+0000 7f2bb4f1e700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ba00a4160 0x7f2ba00a6620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4000c00 tx=0x7f2ba4005fb0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.709+0000 7f2badffb700 1 -- 172.21.15.47:0/558121429 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b9c01f020 con 0x7f2bb0101d10 2024-04-25T21:01:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.965+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bb002df80 con 0x7f2bb0101d10 2024-04-25T21:01:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.966+0000 7f2badffb700 1 -- 172.21.15.47:0/558121429 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11853 (secure 0 0 0) 0x7f2b9c09e280 con 0x7f2bb0101d10 2024-04-25T21:01:25.967 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:01:25.967 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:59:57.345381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:01:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.969+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ba00a4160 msgr2=0x7f2ba00a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.969+0000 7f2bb7182700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ba00a4160 0x7f2ba00a6620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4000c00 tx=0x7f2ba4005fb0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.969+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101d10 msgr2=0x7f2bb00fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.969+0000 7f2bb7182700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101d10 0x7f2bb00fd960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00d950 tx=0x7f2b9c00dc60 comp rx=0 tx=0).stop 2024-04-25T21:01:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.970+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 shutdown_connections 2024-04-25T21:01:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.970+0000 7f2bb7182700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ba00a4160 0x7f2ba00a6620 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.970+0000 7f2bb7182700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb01063f0 0x7f2bb00fdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.970+0000 7f2bb7182700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb0101360 0x7f2bb00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.970+0000 7f2bb7182700 1 --2- 172.21.15.47:0/558121429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bb0101d10 0x7f2bb00fd960 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.970+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 >> 172.21.15.47:0/558121429 conn(0x7f2bb0077fb0 msgr2=0x7f2bb00ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.970+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 shutdown_connections 2024-04-25T21:01:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:25.970+0000 7f2bb7182700 1 -- 172.21.15.47:0/558121429 wait complete. 2024-04-25T21:01:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-04-25T21:01:26.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:26 smithi047 ceph-mon[29195]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:01:26.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:26 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/558121429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:01:26.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:26 smithi186 ceph-mon[32529]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:01:26.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:26 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/558121429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:01:26.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:26 smithi092 ceph-mon[32524]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:01:26.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:26 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/558121429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:01:26.612 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id f79dd869 --release --await' 2024-04-25T21:01:26.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24488 quiesce db --set-id f79dd869 --release --await 2024-04-25T21:01:26.950 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:01:28.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:28 smithi047 ceph-mon[29195]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T21:01:28.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:28 smithi186 ceph-mon[32529]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T21:01:28.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:28 smithi092 ceph-mon[32524]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T21:01:28.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1145850667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf010f4e0 msgr2=0x7fbbf01118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1145850667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf010f4e0 0x7fbbf01118d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800b0a0 tx=0x7fbbe800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1145850667 shutdown_connections 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1145850667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf010f4e0 0x7fbbf01118d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1145850667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf006afe0 0x7fbbf006b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1145850667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf006b900 0x7fbbf010efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1145850667 >> 172.21.15.47:0/1145850667 conn(0x7fbbf00788b0 msgr2=0x7fbbf0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1145850667 shutdown_connections 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.899+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1145850667 wait complete. 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf6ee5700 1 Processor -- start 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf6ee5700 1 -- start start 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf6ee5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf006afe0 0x7fbbf019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf6ee5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf006b900 0x7fbbf019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf6ee5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf010f4e0 0x7fbbf019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf6ee5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0114e60 con 0x7fbbf006afe0 2024-04-25T21:01:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf6ee5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0114ce0 con 0x7fbbf006b900 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf6ee5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0114fe0 con 0x7fbbf010f4e0 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf4c81700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf006afe0 0x7fbbf019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf4c81700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf006afe0 0x7fbbf019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40280/0 (socket says 172.21.15.47:40280) 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf4c81700 1 -- 172.21.15.47:0/1868043429 learned_addr learned my addr 172.21.15.47:0/1868043429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbeffff700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf006b900 0x7fbbf019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.900+0000 7fbbf5482700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf010f4e0 0x7fbbf019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbf4c81700 1 -- 172.21.15.47:0/1868043429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf010f4e0 msgr2=0x7fbbf019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf010f4e0 0x7fbbf019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbf4c81700 1 -- 172.21.15.47:0/1868043429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf006b900 msgr2=0x7fbbf019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf006b900 0x7fbbf019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbf4c81700 1 -- 172.21.15.47:0/1868043429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf01a0360 con 0x7fbbf006afe0 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf006afe0 0x7fbbf019b350 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000d950 tx=0x7fbbe000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbedffb700 1 -- 172.21.15.47:0/1868043429 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0016070 con 0x7fbbf006afe0 2024-04-25T21:01:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1868043429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf01a0650 con 0x7fbbf006afe0 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbedffb700 1 -- 172.21.15.47:0/1868043429 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe0009df0 con 0x7fbbf006afe0 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbedffb700 1 -- 172.21.15.47:0/1868043429 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000f3c0 con 0x7fbbf006afe0 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.901+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1868043429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf01a7ed0 con 0x7fbbf006afe0 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.903+0000 7fbbedffb700 1 -- 172.21.15.47:0/1868043429 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fbbe0019070 con 0x7fbbf006afe0 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.904+0000 7fbbedffb700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbbd80a3f60 0x7fbbd80a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.904+0000 7fbbeffff700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbbd80a3f60 0x7fbbd80a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.904+0000 7fbbedffb700 1 -- 172.21.15.47:0/1868043429 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fbbe00d06b0 con 0x7fbbf006afe0 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.905+0000 7fbbeffff700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbbd80a3f60 0x7fbbd80a6420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0102e00 tx=0x7fbbe4009190 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.906+0000 7fbc0176eb80 1 Processor -- start 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.906+0000 7fbc0176eb80 1 -- start start 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.906+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6650 0x55caf67f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf6801930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 0x55caf6804260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55caf6749d90 con 0x55caf67f6f70 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55caf66f0a30 con 0x55caf67f6650 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55caf6745d50 con 0x55caf6801e70 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf6801930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf6801930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40286/0 (socket says 172.21.15.47:40286) 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbeffff700 1 -- 172.21.15.47:0/386717174 learned_addr learned my addr 172.21.15.47:0/386717174 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6650 0x55caf67f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbf5482700 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 0x55caf6804260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbf4c81700 1 -- 172.21.15.47:0/386717174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 msgr2=0x55caf6804260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 0x55caf6804260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbf4c81700 1 -- 172.21.15.47:0/386717174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 msgr2=0x55caf6801930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf6801930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbf4c81700 1 -- 172.21.15.47:0/386717174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf66b4840 con 0x55caf67f6650 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6650 0x55caf67f6a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00cdbd0 tx=0x7fbbe009cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e0c330eadce67f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbd6ffd700 1 -- 172.21.15.47:0/386717174 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0020070 con 0x55caf67f6650 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbd6ffd700 1 -- 172.21.15.47:0/386717174 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe00a07d0 con 0x55caf67f6650 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.907+0000 7fbbd6ffd700 1 -- 172.21.15.47:0/386717174 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe009b870 con 0x55caf67f6650 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.908+0000 7fbc0176eb80 1 -- 172.21.15.47:0/386717174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6650 msgr2=0x55caf67f6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.908+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6650 0x55caf67f6a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00cdbd0 tx=0x7fbbe009cc10 comp rx=0 tx=0).stop 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.908+0000 7fbc0176eb80 1 -- 172.21.15.47:0/386717174 shutdown_connections 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.908+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 0x55caf6804260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.908+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6650 0x55caf67f6a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.908+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/386717174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf6801930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.908+0000 7fbc0176eb80 1 -- 172.21.15.47:0/386717174 >> 172.21.15.47:0/386717174 conn(0x55caf6723990 msgr2=0x55caf67f2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 -- 172.21.15.47:0/386717174 shutdown_connections 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 -- 172.21.15.47:0/386717174 wait complete. 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 -- start start 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 Processor -- start 2024-04-25T21:01:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6801610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf67fe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 0x55caf67fee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55caf6804840 con 0x55caf67f6f70 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55caf67036b0 con 0x55caf6801e70 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55caf68049c0 con 0x55caf67f6650 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.909+0000 7fbc0176eb80 10 client.0 _ll_register_callbacks cb 0x55caf66e2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf67fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf67fe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40300/0 (socket says 172.21.15.47:40300) 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 -- 172.21.15.47:0/4280106354 learned_addr learned my addr 172.21.15.47:0/4280106354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbf5482700 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 0x55caf67fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6801610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 -- 172.21.15.47:0/4280106354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 msgr2=0x55caf6801610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6801610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 -- 172.21.15.47:0/4280106354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 msgr2=0x55caf67fee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 0x55caf67fee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 -- 172.21.15.47:0/4280106354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf66979d0 con 0x55caf67f6f70 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6801610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbeffff700 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf67fe910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400ef80 tx=0x7fbbe4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf7df144cd1197d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbcffff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbcffff700 1 -- 172.21.15.47:0/4280106354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4025070 con 0x55caf67f6f70 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbc0176eb80 1 -- 172.21.15.47:0/4280106354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe00a07d0 con 0x55caf67f6f70 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.910+0000 7fbbcffff700 1 -- 172.21.15.47:0/4280106354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fbbe4015bd0 con 0x55caf67f6f70 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbbcffff700 1 -- 172.21.15.47:0/4280106354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbe401b040 con 0x55caf67f6f70 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbc0176eb80 10 client.15483 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbc0176eb80 1 -- 172.21.15.47:0/4280106354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55caf680f170 con 0x55caf67f6f70 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbbcffff700 1 -- 172.21.15.47:0/4280106354 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe00a07d0 con 0x55caf67f6f70 2024-04-25T21:01:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbbcffff700 1 -- 172.21.15.47:0/4280106354 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbbe4006580 con 0x55caf67f6f70 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbc0176eb80 10 client.15483 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbc0176eb80 10 client.15483 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbc0176eb80 20 client.15483 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55caf6811700 0x55caf6813b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbc0176eb80 4 client.15483 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbc0176eb80 1 -- 172.21.15.47:0/4280106354 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf6757580 con 0x55caf6811700 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.911+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55caf6811700 0x55caf6813b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.912+0000 7fbbcffff700 10 client.15483 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.912+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55caf6811700 0x55caf6813b50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf6c0500dc04d5a server_cookie=456b2553e513d270 in_seq=0 out_seq=0 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.912+0000 7fbbcffff700 1 -- 172.21.15.47:0/4280106354 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55caf6757580 con 0x55caf6811700 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.912+0000 7fbbcffff700 10 client.15483 handle_command_reply: tid=0 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.912+0000 7fbc0176eb80 1 client.15483 shutdown 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.912+0000 7fbc0176eb80 1 -- 172.21.15.47:0/4280106354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 msgr2=0x55caf67fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.912+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf67fe910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400ef80 tx=0x7fbbe4004860 comp rx=0 tx=0).stop 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 1 -- 172.21.15.47:0/4280106354 shutdown_connections 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55caf6811700 0x55caf6813b50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 0x55caf67fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6801610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/4280106354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf67fe910 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 1 -- 172.21.15.47:0/4280106354 >> 172.21.15.47:0/4280106354 conn(0x55caf6723990 msgr2=0x55caf67f1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbbcffff700 0 client.15483 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 1 -- 172.21.15.47:0/4280106354 shutdown_connections 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 1 -- 172.21.15.47:0/4280106354 wait complete. 2024-04-25T21:01:28.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.913+0000 7fbc0176eb80 20 client.15483 trim_cache size 0 max 16384 2024-04-25T21:01:28.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbc0176eb80 1 Processor -- start 2024-04-25T21:01:28.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbc0176eb80 1 -- start start 2024-04-25T21:01:28.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6819460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf68199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 0x55caf6809c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01b1eb0 con 0x55caf67f6f70 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01af740 con 0x55caf6801e70 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ad5b0 con 0x55caf67f6650 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf68199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf68199a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40302/0 (socket says 172.21.15.47:40302) 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbbeffff700 1 -- 172.21.15.47:0/2859418297 learned_addr learned my addr 172.21.15.47:0/2859418297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6819460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbbf5482700 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 0x55caf6809c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.933+0000 7fbbeffff700 1 -- 172.21.15.47:0/2859418297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 msgr2=0x55caf6819460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.934+0000 7fbbeffff700 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6819460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.934+0000 7fbbeffff700 1 -- 172.21.15.47:0/2859418297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 msgr2=0x55caf6809c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.934+0000 7fbbeffff700 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 0x55caf6809c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.934+0000 7fbbeffff700 1 -- 172.21.15.47:0/2859418297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf67fa510 con 0x55caf67f6f70 2024-04-25T21:01:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.934+0000 7fbbeffff700 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf68199a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400e4b0 tx=0x7fbbe400e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e0ba9460703f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.934+0000 7fbbd67fc700 1 -- 172.21.15.47:0/2859418297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4025040 con 0x55caf67f6f70 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.934+0000 7fbbd67fc700 1 -- 172.21.15.47:0/2859418297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe4015dd0 con 0x55caf67f6f70 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.934+0000 7fbbd67fc700 1 -- 172.21.15.47:0/2859418297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe40216a0 con 0x55caf67f6f70 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 -- 172.21.15.47:0/2859418297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 msgr2=0x55caf68199a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf68199a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400e4b0 tx=0x7fbbe400e5e0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 -- 172.21.15.47:0/2859418297 shutdown_connections 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf6801e70 0x55caf6809c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf67f6650 0x55caf6819460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/2859418297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6f70 0x55caf68199a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 -- 172.21.15.47:0/2859418297 >> 172.21.15.47:0/2859418297 conn(0x55caf6723990 msgr2=0x55caf680c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 -- 172.21.15.47:0/2859418297 shutdown_connections 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 -- 172.21.15.47:0/2859418297 wait complete. 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 -- start start 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.935+0000 7fbc0176eb80 1 Processor -- start 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6650 0x55caf6809e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6f70 0x55caf6806210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbc0176eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 0x55caf6806750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01adc00 con 0x55caf67f6650 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01af070 con 0x55caf67f6f70 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbc0176eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ae2d0 con 0x55caf6801e70 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbc0176eb80 10 client.0 _ll_register_callbacks cb 0x55caf66e2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbbf4c81700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6650 0x55caf6809e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbbf4c81700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6650 0x55caf6809e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40306/0 (socket says 172.21.15.47:40306) 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbbf4c81700 1 -- 172.21.15.47:0/560446811 learned_addr learned my addr 172.21.15.47:0/560446811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6f70 0x55caf6806210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.936+0000 7fbbf5482700 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 0x55caf6806750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbeffff700 1 -- 172.21.15.47:0/560446811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 msgr2=0x55caf6806750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbeffff700 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 0x55caf6806750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbeffff700 1 -- 172.21.15.47:0/560446811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6650 msgr2=0x55caf6809e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbeffff700 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6650 0x55caf6809e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbeffff700 1 -- 172.21.15.47:0/560446811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf6806e00 con 0x55caf67f6f70 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbeffff700 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6f70 0x55caf6806210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55caf6745940 tx=0x7fbbe4013be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da4b6989b0dd5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbcffff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbcffff700 1 -- 172.21.15.47:0/560446811 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4025040 con 0x55caf67f6f70 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbc0176eb80 1 -- 172.21.15.47:0/560446811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe4015dd0 con 0x55caf67f6f70 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbcffff700 1 -- 172.21.15.47:0/560446811 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fbbe4035480 con 0x55caf67f6f70 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbbcffff700 1 -- 172.21.15.47:0/560446811 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbe4006d90 con 0x55caf67f6f70 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbc0176eb80 10 client.25145 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.937+0000 7fbc0176eb80 1 -- 172.21.15.47:0/560446811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55caf6878bc0 con 0x55caf67f6f70 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbbcffff700 1 -- 172.21.15.47:0/560446811 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe4015dd0 con 0x55caf67f6f70 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbbcffff700 1 -- 172.21.15.47:0/560446811 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbbe402a7e0 con 0x55caf67f6f70 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbc0176eb80 10 client.25145 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbc0176eb80 10 client.25145 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbc0176eb80 20 client.25145 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55caf68791b0 0x55caf687b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbc0176eb80 4 client.25145 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f79dd869", "release": true, "await": true}] 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbc0176eb80 1 -- 172.21.15.47:0/560446811 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f79dd869", "release": true, "await": true}) -- 0x55caf6751620 con 0x55caf68791b0 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55caf68791b0 0x55caf687b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbbf4c81700 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55caf68791b0 0x55caf687b5d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84410ccb845cabe server_cookie=58727647a39e6a23 in_seq=0 out_seq=0 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.938+0000 7fbbcffff700 10 client.25145 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbbcffff700 1 -- 172.21.15.47:0/560446811 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55caf6751620 con 0x55caf68791b0 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbbcffff700 10 client.25145 handle_command_reply: tid=0 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 client.25145 shutdown 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 -- 172.21.15.47:0/560446811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6f70 msgr2=0x55caf6806210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6f70 0x55caf6806210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55caf6745940 tx=0x7fbbe4013be0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 -- 172.21.15.47:0/560446811 shutdown_connections 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55caf68791b0 0x55caf687b5d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caf67f6650 0x55caf6809e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55caf6801e70 0x55caf6806750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbbcffff700 0 client.25145 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 --2- 172.21.15.47:0/560446811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caf67f6f70 0x55caf6806210 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 -- 172.21.15.47:0/560446811 >> 172.21.15.47:0/560446811 conn(0x55caf6723990 msgr2=0x55caf680cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 -- 172.21.15.47:0/560446811 shutdown_connections 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 1 -- 172.21.15.47:0/560446811 wait complete. 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.941+0000 7fbc0176eb80 20 client.25145 trim_cache size 0 max 16384 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 26, 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stdout: "leader": 24488, 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stdout: "set_version": 4, 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stdout: "sets": { 2024-04-25T21:01:28.951 INFO:teuthology.orchestra.run.smithi047.stdout: "f79dd869": { 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 4, 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "age_ref": 0.0, 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "state": { 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RELEASED", 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.0 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "timeout": 60.0, 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "expiration": 86.2, 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "members": { 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "file:/": { 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "excluded": false, 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "state": { 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RELEASED", 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.0 2024-04-25T21:01:28.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:01:28.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:01:28.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:01:28.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:01:28.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:01:28.953 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbbd80a3f60 msgr2=0x7fbbd80a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbbd80a3f60 0x7fbbd80a6420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0102e00 tx=0x7fbbe4009190 comp rx=0 tx=0).stop 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf006afe0 msgr2=0x7fbbf019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf006afe0 0x7fbbf019b350 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000d950 tx=0x7fbbe000dc60 comp rx=0 tx=0).stop 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1868043429 shutdown_connections 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbbd80a3f60 0x7fbbd80a6420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf006afe0 0x7fbbf019b350 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf010f4e0 0x7fbbf019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 --2- 172.21.15.47:0/1868043429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf006b900 0x7fbbf019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1868043429 >> 172.21.15.47:0/1868043429 conn(0x7fbbf00788b0 msgr2=0x7fbbf0110990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1868043429 shutdown_connections 2024-04-25T21:01:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:28.942+0000 7fbbf6ee5700 1 -- 172.21.15.47:0/1868043429 wait complete. 2024-04-25T21:01:29.871 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f79dd869', seconds elapsed: 42.2 2024-04-25T21:01:29.871 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.2 2024-04-25T21:01:29.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:01:29.901 INFO:tasks.workunit.client.0.smithi047.stderr:Updating files: 95% (11945/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T21:01:30.136 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:01:30.417 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:30 smithi047 ceph-mon[29195]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T21:01:30.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:30 smithi186 ceph-mon[32529]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T21:01:30.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:30 smithi092 ceph-mon[32524]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T21:01:30.602 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to 'e221b808d285b9d02e54cc10c53c84188e8e41e3'. 2024-04-25T21:01:30.602 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-25T21:01:30.602 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T21:01:30.602 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T21:01:30.602 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T21:01:30.602 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-25T21:01:30.602 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-25T21:01:30.603 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at e221b808d28 qa/suites/fs: add quiescer to the fs suite 2024-04-25T21:01:30.604 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T21:01:30.723 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T21:01:30.726 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T21:01:30.726 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T21:01:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 -- 172.21.15.47:0/3578189701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec103b90 msgr2=0x7ff9ec103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 --2- 172.21.15.47:0/3578189701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec103b90 0x7ff9ec103f70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00b0a0 tx=0x7ff9dc00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 -- 172.21.15.47:0/3578189701 shutdown_connections 2024-04-25T21:01:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 --2- 172.21.15.47:0/3578189701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec103b90 0x7ff9ec103f70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 --2- 172.21.15.47:0/3578189701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ec10cf70 0x7ff9ec10f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 --2- 172.21.15.47:0/3578189701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9ec1044b0 0x7ff9ec10c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 -- 172.21.15.47:0/3578189701 >> 172.21.15.47:0/3578189701 conn(0x7ff9ec0fd360 msgr2=0x7ff9ec0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 -- 172.21.15.47:0/3578189701 shutdown_connections 2024-04-25T21:01:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.164+0000 7ff9f0f9e700 1 -- 172.21.15.47:0/3578189701 wait complete. 2024-04-25T21:01:32.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:32 smithi047 ceph-mon[29195]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T21:01:32.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:32 smithi047 ceph-mon[29195]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T21:01:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.165+0000 7ff9f0f9e700 1 Processor -- start 2024-04-25T21:01:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.165+0000 7ff9f0f9e700 1 -- start start 2024-04-25T21:01:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.165+0000 7ff9f0f9e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9ec103b90 0x7ff9ec198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.165+0000 7ff9f0f9e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ec1044b0 0x7ff9ec1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9f0f9e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec10cf70 0x7ff9ec19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9f0f9e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec112b40 con 0x7ff9ec10cf70 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9f0f9e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1129c0 con 0x7ff9ec1044b0 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9f0f9e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec112cc0 con 0x7ff9ec103b90 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9eaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ec1044b0 0x7ff9ec1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9eaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ec1044b0 0x7ff9ec1994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:53950/0 (socket says 172.21.15.47:53950) 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9eaffd700 1 -- 172.21.15.47:0/1882555722 learned_addr learned my addr 172.21.15.47:0/1882555722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9ebfff700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec10cf70 0x7ff9ec19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9eb7fe700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9ec103b90 0x7ff9ec198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9ebfff700 1 -- 172.21.15.47:0/1882555722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9ec103b90 msgr2=0x7ff9ec198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9ebfff700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9ec103b90 0x7ff9ec198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9ebfff700 1 -- 172.21.15.47:0/1882555722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ec1044b0 msgr2=0x7ff9ec1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9ebfff700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ec1044b0 0x7ff9ec1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9ebfff700 1 -- 172.21.15.47:0/1882555722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec19dfc0 con 0x7ff9ec10cf70 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9eb7fe700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9ec103b90 0x7ff9ec198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9eaffd700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ec1044b0 0x7ff9ec1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.166+0000 7ff9ebfff700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec10cf70 0x7ff9ec19d880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400c970 tx=0x7ff9e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.167+0000 7ff9e8ff9700 1 -- 172.21.15.47:0/1882555722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400e880 con 0x7ff9ec10cf70 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.167+0000 7ff9f0f9e700 1 -- 172.21.15.47:0/1882555722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec19e2b0 con 0x7ff9ec10cf70 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.167+0000 7ff9e8ff9700 1 -- 172.21.15.47:0/1882555722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e400f450 con 0x7ff9ec10cf70 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.167+0000 7ff9e8ff9700 1 -- 172.21.15.47:0/1882555722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4018660 con 0x7ff9ec10cf70 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.167+0000 7ff9f0f9e700 1 -- 172.21.15.47:0/1882555722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec101720 con 0x7ff9ec10cf70 2024-04-25T21:01:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.168+0000 7ff9d27fc700 1 -- 172.21.15.47:0/1882555722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9cc005330 con 0x7ff9ec10cf70 2024-04-25T21:01:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.169+0000 7ff9e8ff9700 1 -- 172.21.15.47:0/1882555722 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff9e400ea20 con 0x7ff9ec10cf70 2024-04-25T21:01:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.170+0000 7ff9e8ff9700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9d40a3db0 0x7ff9d40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.170+0000 7ff9eb7fe700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9d40a3db0 0x7ff9d40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.170+0000 7ff9e8ff9700 1 -- 172.21.15.47:0/1882555722 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ff9e40d05f0 con 0x7ff9ec10cf70 2024-04-25T21:01:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.170+0000 7ff9eb7fe700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9d40a3db0 0x7ff9d40a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0095c0 tx=0x7ff9dc00b7b0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.173+0000 7ff9e8ff9700 1 -- 172.21.15.47:0/1882555722 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9e4014030 con 0x7ff9ec10cf70 2024-04-25T21:01:32.363 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T21:01:32.427 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T21:01:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.431+0000 7ff9d27fc700 1 -- 172.21.15.47:0/1882555722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9cc005620 con 0x7ff9ec10cf70 2024-04-25T21:01:32.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.432+0000 7ff9e8ff9700 1 -- 172.21.15.47:0/1882555722 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11853 (secure 0 0 0) 0x7ff9e409cd30 con 0x7ff9ec10cf70 2024-04-25T21:01:32.432 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:01:32.432 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T20:59:57.345381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:01:32.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 -- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9d40a3db0 msgr2=0x7ff9d40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9d40a3db0 0x7ff9d40a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0095c0 tx=0x7ff9dc00b7b0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 -- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec10cf70 msgr2=0x7ff9ec19d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec10cf70 0x7ff9ec19d880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400c970 tx=0x7ff9e400cc80 comp rx=0 tx=0).stop 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 -- 172.21.15.47:0/1882555722 shutdown_connections 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9d40a3db0 0x7ff9d40a6270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9ec10cf70 0x7ff9ec19d880 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9ec103b90 0x7ff9ec198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/1882555722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ec1044b0 0x7ff9ec1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 -- 172.21.15.47:0/1882555722 >> 172.21.15.47:0/1882555722 conn(0x7ff9ec0fd360 msgr2=0x7ff9ec0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 -- 172.21.15.47:0/1882555722 shutdown_connections 2024-04-25T21:01:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:32.434+0000 7ff9d27fc700 1 -- 172.21.15.47:0/1882555722 wait complete. 2024-04-25T21:01:32.436 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-04-25T21:01:32.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:32 smithi186 ceph-mon[32529]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T21:01:32.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:32 smithi186 ceph-mon[32529]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T21:01:32.473 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T21:01:32.476 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T21:01:32.476 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T21:01:32.520 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T21:01:32.523 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T21:01:32.523 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T21:01:32.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:32 smithi092 ceph-mon[32524]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T21:01:32.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:32 smithi092 ceph-mon[32524]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T21:01:32.539 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-25T21:01:32.541 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-25T21:01:32.541 DEBUG:teuthology.orchestra.run.smithi047:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e221b808d285b9d02e54cc10c53c84188e8e41e3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-25T21:01:32.602 INFO:tasks.workunit.client.0.smithi047.stderr:+ set -e 2024-04-25T21:01:32.602 INFO:tasks.workunit.client.0.smithi047.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-25T21:01:32.604 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into 'xfstests-dev'... 2024-04-25T21:01:33.209 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0' 2024-04-25T21:01:33.210 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24488 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0 2024-04-25T21:01:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:33 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1882555722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:01:33.499 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:01:33.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:33 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1882555722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:01:33.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:33 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1882555722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 -- 172.21.15.47:0/572991802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 msgr2=0x7f8f701014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 --2- 172.21.15.47:0/572991802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 0x7f8f701014d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009900 tx=0x7f8f6002a3f0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 -- 172.21.15.47:0/572991802 shutdown_connections 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 --2- 172.21.15.47:0/572991802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f70106170 0x7f8f7010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 --2- 172.21.15.47:0/572991802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f70101aa0 0x7f8f70105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 --2- 172.21.15.47:0/572991802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 0x7f8f701014d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 -- 172.21.15.47:0/572991802 >> 172.21.15.47:0/572991802 conn(0x7f8f70077fd0 msgr2=0x7f8f700783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 -- 172.21.15.47:0/572991802 shutdown_connections 2024-04-25T21:01:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.840+0000 7f8f74a05640 1 -- 172.21.15.47:0/572991802 wait complete. 2024-04-25T21:01:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f74a05640 1 Processor -- start 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f74a05640 1 -- start start 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f74a05640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 0x7f8f701a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f74a05640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f70106170 0x7f8f701a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f74a05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f7019a6b0 0x7f8f7019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f74a05640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f701121f0 con 0x7f8f701010f0 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f74a05640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f70112070 con 0x7f8f7019a6b0 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f74a05640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8f70112370 con 0x7f8f70106170 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f6e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 0x7f8f701a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f6e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 0x7f8f701a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38668/0 (socket says 172.21.15.47:38668) 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.841+0000 7f8f6e59c640 1 -- 172.21.15.47:0/3189843379 learned_addr learned my addr 172.21.15.47:0/3189843379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f70106170 0x7f8f701a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f7019a6b0 0x7f8f7019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6e59c640 1 -- 172.21.15.47:0/3189843379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f70106170 msgr2=0x7f8f701a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f70106170 0x7f8f701a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6e59c640 1 -- 172.21.15.47:0/3189843379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f7019a6b0 msgr2=0x7f8f7019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f7019a6b0 0x7f8f7019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6e59c640 1 -- 172.21.15.47:0/3189843379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7019b2b0 con 0x7f8f701010f0 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f7019a6b0 0x7f8f7019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 0x7f8f701a05f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8f60005940 tx=0x7f8f600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f5f7fe640 1 -- 172.21.15.47:0/3189843379 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6003f030 con 0x7f8f701010f0 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f5f7fe640 1 -- 172.21.15.47:0/3189843379 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f60003cb0 con 0x7f8f701010f0 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f74a05640 1 -- 172.21.15.47:0/3189843379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7019b540 con 0x7f8f701010f0 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f5f7fe640 1 -- 172.21.15.47:0/3189843379 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60035070 con 0x7f8f701010f0 2024-04-25T21:01:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.842+0000 7f8f74a05640 1 -- 172.21.15.47:0/3189843379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f701a76d0 con 0x7f8f701010f0 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f5f7fe640 1 -- 172.21.15.47:0/3189843379 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8f60042030 con 0x7f8f701010f0 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f777f8740 1 Processor -- start 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f777f8740 1 -- start start 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed21a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed2c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eed2ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f777f8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5637eec86970 con 0x5637eed2c800 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f777f8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5637eec234a0 con 0x5637eed21f40 2024-04-25T21:01:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.845+0000 7f8f777f8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5637eec82930 con 0x5637eed21620 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eed2ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed2c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eed2ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38680/0 (socket says 172.21.15.47:38680) 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/2331632794 learned_addr learned my addr 172.21.15.47:0/2331632794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed21a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f5f7fe640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f8f580a3e40 0x7f8f580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/2331632794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 msgr2=0x5637eed21a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed21a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/2331632794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 msgr2=0x5637eed2c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed2c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/2331632794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ee999210 con 0x5637eed2c800 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed2c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed21a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f8f580a3e40 0x7f8f580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f5f7fe640 1 -- 172.21.15.47:0/3189843379 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f8f600f0fe0 con 0x7f8f701010f0 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eed2ebf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b6c0 tx=0x7f8f6400bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e378f384a7f734 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.846+0000 7f8f5cff9640 1 -- 172.21.15.47:0/2331632794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64003bd0 con 0x5637eed2c800 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f8f580a3e40 0x7f8f580a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005cc0 tx=0x7f8f68004a40 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f5cff9640 1 -- 172.21.15.47:0/2331632794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f64015070 con 0x5637eed2c800 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f5cff9640 1 -- 172.21.15.47:0/2331632794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64004d10 con 0x5637eed2c800 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 -- 172.21.15.47:0/2331632794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 msgr2=0x5637eed2ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eed2ebf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b6c0 tx=0x7f8f6400bb80 comp rx=0 tx=0).stop 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 -- 172.21.15.47:0/2331632794 shutdown_connections 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eed2ebf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed2c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2331632794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed21a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 -- 172.21.15.47:0/2331632794 >> 172.21.15.47:0/2331632794 conn(0x5637eed1db30 msgr2=0x5637eed1df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 -- 172.21.15.47:0/2331632794 shutdown_connections 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.847+0000 7f8f777f8740 1 -- 172.21.15.47:0/2331632794 wait complete. 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 1 -- start start 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 1 Processor -- start 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed2c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed29460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed2c800 0x5637eed299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5637eed2f430 con 0x5637eed21f40 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5637eec210b0 con 0x5637eed2c800 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5637eed2f5b0 con 0x5637eed21620 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f777f8740 10 client.0 _ll_register_callbacks cb 0x5637eec0ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f6dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed29460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f6dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed29460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38692/0 (socket says 172.21.15.47:38692) 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/2709347581 learned_addr learned my addr 172.21.15.47:0/2709347581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed2c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.848+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed2c800 0x5637eed299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/2709347581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 msgr2=0x5637eed2c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed2c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/2709347581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed2c800 msgr2=0x5637eed299c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed2c800 0x5637eed299c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/2709347581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637eea65620 con 0x5637eed21f40 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed29460 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8f640048f0 tx=0x7f8f64003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5488103e12e4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f55ffb640 1 -- 172.21.15.47:0/2709347581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64028060 con 0x5637eed21f40 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f55ffb640 1 -- 172.21.15.47:0/2709347581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f8f64011650 con 0x5637eed21f40 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f777f8740 1 -- 172.21.15.47:0/2709347581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f64015070 con 0x5637eed21f40 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f55ffb640 1 -- 172.21.15.47:0/2709347581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f64011850 con 0x5637eed21f40 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f777f8740 10 client.15507 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f777f8740 1 -- 172.21.15.47:0/2709347581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637eed3b360 con 0x5637eed21f40 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.849+0000 7f8f55ffb640 1 -- 172.21.15.47:0/2709347581 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f64015070 con 0x5637eed21f40 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f55ffb640 1 -- 172.21.15.47:0/2709347581 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8f64022070 con 0x5637eed21f40 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f777f8740 10 client.15507 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f777f8740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f777f8740 20 client.15507 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5637eed3c760 0x5637eed3eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f777f8740 4 client.15507 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f777f8740 1 -- 172.21.15.47:0/2709347581 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637eec94390 con 0x5637eed3c760 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5637eed3c760 0x5637eed3eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f55ffb640 10 client.15507 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.850+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5637eed3c760 0x5637eed3eb50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83d6e18711bcae1 server_cookie=323ac0859580ade9 in_seq=0 out_seq=0 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f55ffb640 1 -- 172.21.15.47:0/2709347581 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637eec94390 con 0x5637eed3c760 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f55ffb640 10 client.15507 handle_command_reply: tid=0 2024-04-25T21:01:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 client.15507 shutdown 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 -- 172.21.15.47:0/2709347581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 msgr2=0x5637eed29460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed29460 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8f640048f0 tx=0x7f8f64003e00 comp rx=0 tx=0).stop 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 -- 172.21.15.47:0/2709347581 shutdown_connections 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5637eed3c760 0x5637eed3eb50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed2c800 0x5637eed299c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed29460 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 --2- 172.21.15.47:0/2709347581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed2c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 -- 172.21.15.47:0/2709347581 >> 172.21.15.47:0/2709347581 conn(0x5637eed1db30 msgr2=0x5637eed23860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f55ffb640 0 client.15507 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 -- 172.21.15.47:0/2709347581 shutdown_connections 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.851+0000 7f8f777f8740 1 -- 172.21.15.47:0/2709347581 wait complete. 2024-04-25T21:01:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.852+0000 7f8f777f8740 20 client.15507 trim_cache size 0 max 16384 2024-04-25T21:01:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.879+0000 7f8f777f8740 1 Processor -- start 2024-04-25T21:01:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.879+0000 7f8f777f8740 1 -- start start 2024-04-25T21:01:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21620 0x5637eed382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed3e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed2c800 0x5637eed2bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f777f8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f701b24a0 con 0x5637eed21f40 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f777f8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f701afd70 con 0x5637eed21620 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f777f8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8f701adbe0 con 0x5637eed2c800 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed3e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed3e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38702/0 (socket says 172.21.15.47:38702) 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/3988773926 learned_addr learned my addr 172.21.15.47:0/3988773926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21620 0x5637eed382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed2c800 0x5637eed2bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/3988773926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed2c800 msgr2=0x5637eed2bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed2c800 0x5637eed2bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/3988773926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21620 msgr2=0x5637eed382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21620 0x5637eed382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 -- 172.21.15.47:0/3988773926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637eed279f0 con 0x5637eed21f40 2024-04-25T21:01:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed3e770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8f64004920 tx=0x7f8f64005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb984bcdc8f50c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f57fff640 1 -- 172.21.15.47:0/3988773926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64028040 con 0x5637eed21f40 2024-04-25T21:01:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f57fff640 1 -- 172.21.15.47:0/3988773926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f64015070 con 0x5637eed21f40 2024-04-25T21:01:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.880+0000 7f8f57fff640 1 -- 172.21.15.47:0/3988773926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64011e50 con 0x5637eed21f40 2024-04-25T21:01:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 -- 172.21.15.47:0/3988773926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 msgr2=0x5637eed3e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed3e770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8f64004920 tx=0x7f8f64005c50 comp rx=0 tx=0).stop 2024-04-25T21:01:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 -- 172.21.15.47:0/3988773926 shutdown_connections 2024-04-25T21:01:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed2c800 0x5637eed2bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed21f40 0x5637eed3e770 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 --2- 172.21.15.47:0/3988773926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21620 0x5637eed382f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 -- 172.21.15.47:0/3988773926 >> 172.21.15.47:0/3988773926 conn(0x5637eed1db30 msgr2=0x5637eed249d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 -- 172.21.15.47:0/3988773926 shutdown_connections 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.881+0000 7f8f777f8740 1 -- 172.21.15.47:0/3988773926 wait complete. 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 1 -- start start 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 1 Processor -- start 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eeda19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f701ae230 con 0x5637eed2c800 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f701af6a0 con 0x5637eed21f40 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8f701ae900 con 0x5637eed21620 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f777f8740 10 client.0 _ll_register_callbacks cb 0x7f8f64004290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eeda19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eeda19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38706/0 (socket says 172.21.15.47:38706) 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6ed9d640 1 -- 172.21.15.47:0/4165910223 learned_addr learned my addr 172.21.15.47:0/4165910223 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6dd9b640 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6ed9d640 1 -- 172.21.15.47:0/4165910223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 msgr2=0x5637eed26840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed26840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6ed9d640 1 -- 172.21.15.47:0/4165910223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 msgr2=0x5637eed26d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed26d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.882+0000 7f8f6ed9d640 1 -- 172.21.15.47:0/4165910223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637eeda1f30 con 0x5637eed2c800 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f6ed9d640 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eeda19f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8f680119c0 tx=0x7f8f680084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdf998a62381a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f55ffb640 1 -- 172.21.15.47:0/4165910223 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6802b060 con 0x5637eed2c800 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f55ffb640 1 -- 172.21.15.47:0/4165910223 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f8f6801ad40 con 0x5637eed2c800 2024-04-25T21:01:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f777f8740 1 -- 172.21.15.47:0/4165910223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f64015070 con 0x5637eed2c800 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f55ffb640 1 -- 172.21.15.47:0/4165910223 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f6801c460 con 0x5637eed2c800 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f777f8740 10 client.15513 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f777f8740 1 -- 172.21.15.47:0/4165910223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637eeda3250 con 0x5637eed2c800 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f55ffb640 1 -- 172.21.15.47:0/4165910223 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f64015070 con 0x5637eed2c800 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f55ffb640 1 -- 172.21.15.47:0/4165910223 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8f68025030 con 0x5637eed2c800 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f777f8740 10 client.15513 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f777f8740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.883+0000 7f8f777f8740 20 client.15513 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.884+0000 7f8f777f8740 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5637eeda39d0 0x5637eeda5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.884+0000 7f8f777f8740 4 client.15513 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.884+0000 7f8f777f8740 1 -- 172.21.15.47:0/4165910223 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637eec8e200 con 0x5637eeda39d0 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.884+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5637eeda39d0 0x5637eeda5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.884+0000 7f8f6e59c640 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5637eeda39d0 0x5637eeda5df0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd19603f459b7f5 server_cookie=eff7228dd4226f54 in_seq=0 out_seq=0 2024-04-25T21:01:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.884+0000 7f8f55ffb640 10 client.15513 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f55ffb640 1 -- 172.21.15.47:0/4165910223 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637eec8e200 con 0x5637eeda39d0 2024-04-25T21:01:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f55ffb640 10 client.15513 handle_command_reply: tid=0 2024-04-25T21:01:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 client.15513 shutdown 2024-04-25T21:01:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 -- 172.21.15.47:0/4165910223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 msgr2=0x5637eeda19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eeda19f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8f680119c0 tx=0x7f8f680084c0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 -- 172.21.15.47:0/4165910223 shutdown_connections 2024-04-25T21:01:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5637eeda39d0 0x5637eeda5df0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637eed2c800 0x5637eeda19f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637eed21f40 0x5637eed26d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 --2- 172.21.15.47:0/4165910223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637eed21620 0x5637eed26840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 -- 172.21.15.47:0/4165910223 >> 172.21.15.47:0/4165910223 conn(0x5637eed1db30 msgr2=0x5637eed1c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f55ffb640 0 client.15513 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 -- 172.21.15.47:0/4165910223 shutdown_connections 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 1 -- 172.21.15.47:0/4165910223 wait complete. 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.905+0000 7f8f777f8740 20 client.15513 trim_cache size 0 max 16384 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:01:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:01:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:01:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 -- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f8f580a3e40 msgr2=0x7f8f580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f8f580a3e40 0x7f8f580a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005cc0 tx=0x7f8f68004a40 comp rx=0 tx=0).stop 2024-04-25T21:01:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 -- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 msgr2=0x7f8f701a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 0x7f8f701a05f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8f60005940 tx=0x7f8f600042f0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 -- 172.21.15.47:0/3189843379 shutdown_connections 2024-04-25T21:01:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f8f580a3e40 0x7f8f580a6300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f7019a6b0 0x7f8f7019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f70106170 0x7f8f701a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 --2- 172.21.15.47:0/3189843379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f701010f0 0x7f8f701a05f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 -- 172.21.15.47:0/3189843379 >> 172.21.15.47:0/3189843379 conn(0x7f8f70077fd0 msgr2=0x7f8f700fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 -- 172.21.15.47:0/3189843379 shutdown_connections 2024-04-25T21:01:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:33.906+0000 7f8f74a05640 1 -- 172.21.15.47:0/3189843379 wait complete. 2024-04-25T21:01:34.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:34 smithi047 ceph-mon[29195]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T21:01:34.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:34 smithi186 ceph-mon[32529]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T21:01:34.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:34 smithi092 ceph-mon[32524]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T21:01:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 -- 172.21.15.47:0/368556797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 msgr2=0x7f6a98105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 --2- 172.21.15.47:0/368556797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 0x7f6a98105d10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0099c0 tx=0x7f6a8c009cd0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 -- 172.21.15.47:0/368556797 shutdown_connections 2024-04-25T21:01:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 --2- 172.21.15.47:0/368556797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a981063f0 0x7f6a9810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 --2- 172.21.15.47:0/368556797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a98101310 0x7f6a981016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 --2- 172.21.15.47:0/368556797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 0x7f6a98105d10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 -- 172.21.15.47:0/368556797 >> 172.21.15.47:0/368556797 conn(0x7f6a98077fe0 msgr2=0x7f6a980ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 -- 172.21.15.47:0/368556797 shutdown_connections 2024-04-25T21:01:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.183+0000 7f6a9e451700 1 -- 172.21.15.47:0/368556797 wait complete. 2024-04-25T21:01:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a9e451700 1 Processor -- start 2024-04-25T21:01:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a9e451700 1 -- start start 2024-04-25T21:01:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a9e451700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a98101310 0x7f6a9819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a9e451700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 0x7f6a9819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a9e451700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a981063f0 0x7f6a9819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a9e451700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981109d0 con 0x7f6a98101cc0 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a9e451700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a98110850 con 0x7f6a981063f0 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a9e451700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a98110b50 con 0x7f6a98101310 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.184+0000 7f6a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 0x7f6a9819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 0x7f6a9819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38716/0 (socket says 172.21.15.47:38716) 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a977fe700 1 -- 172.21.15.47:0/2380327434 learned_addr learned my addr 172.21.15.47:0/2380327434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a981063f0 0x7f6a9819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a97fff700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a98101310 0x7f6a9819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9c9ee700 1 -- 172.21.15.47:0/2380327434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a98101310 msgr2=0x7f6a9819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a98101310 0x7f6a9819b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9c9ee700 1 -- 172.21.15.47:0/2380327434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 msgr2=0x7f6a9819b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 0x7f6a9819b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9c9ee700 1 -- 172.21.15.47:0/2380327434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a981a0220 con 0x7f6a981063f0 2024-04-25T21:01:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a977fe700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a98101cc0 0x7f6a9819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a981063f0 0x7f6a9819fae0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a90009f00 tx=0x7f6a900076c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a957fa700 1 -- 172.21.15.47:0/2380327434 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90019070 con 0x7f6a981063f0 2024-04-25T21:01:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9e451700 1 -- 172.21.15.47:0/2380327434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a981a0510 con 0x7f6a981063f0 2024-04-25T21:01:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a957fa700 1 -- 172.21.15.47:0/2380327434 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a90003c00 con 0x7f6a981063f0 2024-04-25T21:01:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a957fa700 1 -- 172.21.15.47:0/2380327434 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90014650 con 0x7f6a981063f0 2024-04-25T21:01:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.185+0000 7f6a9e451700 1 -- 172.21.15.47:0/2380327434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a981a7d90 con 0x7f6a981063f0 2024-04-25T21:01:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.188+0000 7f6a957fa700 1 -- 172.21.15.47:0/2380327434 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6a9001c070 con 0x7f6a981063f0 2024-04-25T21:01:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.188+0000 7f6a957fa700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6a800a3e50 0x7f6a800a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6a97fff700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6a800a3e50 0x7f6a800a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6a957fa700 1 -- 172.21.15.47:0/2380327434 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6a900d0130 con 0x7f6a981063f0 2024-04-25T21:01:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6aa8cdab80 1 Processor -- start 2024-04-25T21:01:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6aa8cdab80 1 -- start start 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6a97fff700 1 --2- 172.21.15.47:0/2380327434 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6a800a3e50 0x7f6a800a6310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a88005950 tx=0x7f6a8800b410 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6be6e0 0x56076c6beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6bf000 0x56076c6c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56076c611e20 con 0x56076c6be6e0 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56076c5b8ac0 con 0x56076c6c9f00 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56076c60dde0 con 0x56076c6bf000 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.189+0000 7f6a97fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6be6e0 0x56076c6beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a97fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6be6e0 0x56076c6beac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38728/0 (socket says 172.21.15.47:38728) 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a977fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6bf000 0x56076c6c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a97fff700 1 -- 172.21.15.47:0/1119488985 learned_addr learned my addr 172.21.15.47:0/1119488985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a97fff700 1 -- 172.21.15.47:0/1119488985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6bf000 msgr2=0x56076c6c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a97fff700 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6bf000 0x56076c6c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a97fff700 1 -- 172.21.15.47:0/1119488985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 msgr2=0x56076c6cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:35.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a97fff700 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a97fff700 1 -- 172.21.15.47:0/1119488985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076c57c8d0 con 0x56076c6be6e0 2024-04-25T21:01:35.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a977fe700 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6bf000 0x56076c6c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:35.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a97fff700 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6be6e0 0x56076c6beac0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6a88010fa0 tx=0x7f6a88013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef2786c05b6b643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:35.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a7e7fc700 1 -- 172.21.15.47:0/1119488985 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800be60 con 0x56076c6be6e0 2024-04-25T21:01:35.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a7e7fc700 1 -- 172.21.15.47:0/1119488985 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a88011db0 con 0x56076c6be6e0 2024-04-25T21:01:35.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.190+0000 7f6a7e7fc700 1 -- 172.21.15.47:0/1119488985 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8801c700 con 0x56076c6be6e0 2024-04-25T21:01:35.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1119488985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6be6e0 msgr2=0x56076c6beac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:35.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6be6e0 0x56076c6beac0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6a88010fa0 tx=0x7f6a88013000 comp rx=0 tx=0).stop 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1119488985 shutdown_connections 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6be6e0 0x56076c6beac0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1119488985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6bf000 0x56076c6c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1119488985 >> 172.21.15.47:0/1119488985 conn(0x56076c5eba20 msgr2=0x56076c6baba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1119488985 shutdown_connections 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.191+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1119488985 wait complete. 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 1 -- start start 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 1 Processor -- start 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6be6e0 0x56076c6c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6c9f00 0x56076c6c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56076c6cc990 con 0x56076c6bf000 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56076c5cb740 con 0x56076c6be6e0 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56076c6ccb10 con 0x56076c6c9f00 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6aa8cdab80 10 client.0 _ll_register_callbacks cb 0x56076c5aaab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6a97fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6be6e0 0x56076c6c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6a97fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6be6e0 0x56076c6c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:38934/0 (socket says 172.21.15.47:38934) 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6a97fff700 1 -- 172.21.15.47:0/1489290821 learned_addr learned my addr 172.21.15.47:0/1489290821 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.192+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6c9f00 0x56076c6c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a977fe700 1 -- 172.21.15.47:0/1489290821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6c9f00 msgr2=0x56076c6c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a977fe700 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6c9f00 0x56076c6c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a977fe700 1 -- 172.21.15.47:0/1489290821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6be6e0 msgr2=0x56076c6c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a977fe700 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6be6e0 0x56076c6c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a977fe700 1 -- 172.21.15.47:0/1489290821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56076c55fa60 con 0x56076c6bf000 2024-04-25T21:01:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6c9f00 0x56076c6c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a97fff700 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6be6e0 0x56076c6c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a977fe700 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6c69b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c003760 tx=0x7f6a8c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b60b496e673e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a777fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a777fe700 1 -- 172.21.15.47:0/1489290821 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0037c0 con 0x56076c6bf000 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a777fe700 1 -- 172.21.15.47:0/1489290821 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6a8c003960 con 0x56076c6bf000 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1489290821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a88011db0 con 0x56076c6bf000 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a777fe700 1 -- 172.21.15.47:0/1489290821 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c032050 con 0x56076c6bf000 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6aa8cdab80 10 client.15525 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1489290821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56076c6d7270 con 0x56076c6bf000 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.193+0000 7f6a777fe700 1 -- 172.21.15.47:0/1489290821 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a88011db0 con 0x56076c6bf000 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6a777fe700 1 -- 172.21.15.47:0/1489290821 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6a8c00fbb0 con 0x56076c6bf000 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6aa8cdab80 10 client.15525 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:35.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6aa8cdab80 10 client.15525 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:01:35.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6aa8cdab80 20 client.15525 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:35.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56076c6d98b0 0x56076c6dbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6aa8cdab80 4 client.15525 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:35.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1489290821 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56076c61f610 con 0x56076c6d98b0 2024-04-25T21:01:35.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6a97fff700 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56076c6d98b0 0x56076c6dbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6a777fe700 10 client.15525 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:35.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.194+0000 7f6a97fff700 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56076c6d98b0 0x56076c6dbca0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4f99bd54b1283e server_cookie=b0799a50bd208efa in_seq=0 out_seq=0 2024-04-25T21:01:35.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.195+0000 7f6a777fe700 1 -- 172.21.15.47:0/1489290821 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56076c61f610 con 0x56076c6d98b0 2024-04-25T21:01:35.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.195+0000 7f6a777fe700 10 client.15525 handle_command_reply: tid=0 2024-04-25T21:01:35.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 client.15525 shutdown 2024-04-25T21:01:35.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1489290821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 msgr2=0x56076c6c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6c69b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c003760 tx=0x7f6a8c005050 comp rx=0 tx=0).stop 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1489290821 shutdown_connections 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56076c6d98b0 0x56076c6dbca0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6c9f00 0x56076c6c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6be6e0 0x56076c6c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6a777fe700 0 client.15525 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/1489290821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6c69b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1489290821 >> 172.21.15.47:0/1489290821 conn(0x56076c5eba20 msgr2=0x56076c6c1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1489290821 shutdown_connections 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/1489290821 wait complete. 2024-04-25T21:01:35.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.196+0000 7f6aa8cdab80 20 client.15525 trim_cache size 0 max 16384 2024-04-25T21:01:35.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6aa8cdab80 1 Processor -- start 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6aa8cdab80 1 -- start start 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 0x56076c6e1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981b1b90 con 0x56076c6bf000 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981af420 con 0x56076c6c9f00 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981ad290 con 0x56076c6be6e0 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6e1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38742/0 (socket says 172.21.15.47:38742) 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a977fe700 1 -- 172.21.15.47:0/3910441818 learned_addr learned my addr 172.21.15.47:0/3910441818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a97fff700 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 0x56076c6e1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a977fe700 1 -- 172.21.15.47:0/3910441818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 msgr2=0x56076c6e1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:35.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a977fe700 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 0x56076c6e1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a977fe700 1 -- 172.21.15.47:0/3910441818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 msgr2=0x56076c6c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a977fe700 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.216+0000 7f6a977fe700 1 -- 172.21.15.47:0/3910441818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076c6c0140 con 0x56076c6bf000 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.217+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.217+0000 7f6a977fe700 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6e1a90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009350 tx=0x7f6a8c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5460877a493c360f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.217+0000 7f6a7dffb700 1 -- 172.21.15.47:0/3910441818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c005540 con 0x56076c6bf000 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.217+0000 7f6a7dffb700 1 -- 172.21.15.47:0/3910441818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c03a030 con 0x56076c6bf000 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.217+0000 7f6a7dffb700 1 -- 172.21.15.47:0/3910441818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c003bb0 con 0x56076c6bf000 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.217+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/3910441818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 msgr2=0x56076c6e1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.217+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6e1a90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009350 tx=0x7f6a8c005e80 comp rx=0 tx=0).stop 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/3910441818 shutdown_connections 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6c9f00 0x56076c6c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 0x56076c6e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/3910441818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6bf000 0x56076c6e1a90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/3910441818 >> 172.21.15.47:0/3910441818 conn(0x56076c5eba20 msgr2=0x56076c6d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/3910441818 shutdown_connections 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/3910441818 wait complete. 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 -- start start 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 Processor -- start 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 0x56076c6d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6bf000 0x56076c6d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6c9f00 0x56076c6d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981ad8e0 con 0x56076c6c9f00 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981aed50 con 0x56076c6bf000 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.218+0000 7f6aa8cdab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981adfb0 con 0x56076c6be6e0 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6aa8cdab80 10 client.0 _ll_register_callbacks cb 0x56076c5aaab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6c9f00 0x56076c6d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6c9f00 0x56076c6d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38752/0 (socket says 172.21.15.47:38752) 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 -- 172.21.15.47:0/2636197440 learned_addr learned my addr 172.21.15.47:0/2636197440 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a97fff700 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 0x56076c6d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a977fe700 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6bf000 0x56076c6d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 -- 172.21.15.47:0/2636197440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 msgr2=0x56076c6d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 0x56076c6d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 -- 172.21.15.47:0/2636197440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6bf000 msgr2=0x56076c6d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6bf000 0x56076c6d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 -- 172.21.15.47:0/2636197440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56076c6d0c50 con 0x56076c6c9f00 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a9c9ee700 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6c9f00 0x56076c6d0710 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6a900cd940 tx=0x7f6a9009b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e433b0c86bee26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a777fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a777fe700 1 -- 172.21.15.47:0/2636197440 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a900b4020 con 0x56076c6c9f00 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a777fe700 1 -- 172.21.15.47:0/2636197440 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6a900a4a20 con 0x56076c6c9f00 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/2636197440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c03a030 con 0x56076c6c9f00 2024-04-25T21:01:35.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6a777fe700 1 -- 172.21.15.47:0/2636197440 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a9009c040 con 0x56076c6c9f00 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.219+0000 7f6aa8cdab80 10 client.15531 fetch_fsmap learned FSMap version 26 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/2636197440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56076c740b00 con 0x56076c6c9f00 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6a777fe700 1 -- 172.21.15.47:0/2636197440 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c03a030 con 0x56076c6c9f00 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6a777fe700 1 -- 172.21.15.47:0/2636197440 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6a90013070 con 0x56076c6c9f00 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6aa8cdab80 10 client.15531 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6aa8cdab80 10 client.15531 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6aa8cdab80 20 client.15531 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56076c741280 0x56076c743670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6aa8cdab80 4 client.15531 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}] 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/2636197440 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}) -- 0x56076c6196b0 con 0x56076c741280 2024-04-25T21:01:35.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.220+0000 7f6a97fff700 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56076c741280 0x56076c743670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:35.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.221+0000 7f6a97fff700 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56076c741280 0x56076c743670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999a97bf825944f5 server_cookie=52848787abaadee8 in_seq=0 out_seq=0 2024-04-25T21:01:35.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:01:35.221+0000 7f6a777fe700 10 client.15531 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:36.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:36 smithi186 ceph-mon[32529]: pgmap v51: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 23 op/s 2024-04-25T21:01:36.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:36 smithi092 ceph-mon[32524]: pgmap v51: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 23 op/s 2024-04-25T21:01:36.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:36 smithi047 ceph-mon[29195]: pgmap v51: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 23 op/s 2024-04-25T21:01:38.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:38 smithi186 ceph-mon[32529]: pgmap v52: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.1 MiB/s wr, 20 op/s 2024-04-25T21:01:38.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:38 smithi092 ceph-mon[32524]: pgmap v52: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.1 MiB/s wr, 20 op/s 2024-04-25T21:01:38.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:38 smithi047 ceph-mon[29195]: pgmap v52: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.1 MiB/s wr, 20 op/s 2024-04-25T21:01:39.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:39 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:01:39.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:39 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:01:39.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:39 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:01:40.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:40 smithi186 ceph-mon[32529]: pgmap v53: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.1 MiB/s wr, 20 op/s 2024-04-25T21:01:40.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:40 smithi092 ceph-mon[32524]: pgmap v53: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.1 MiB/s wr, 20 op/s 2024-04-25T21:01:40.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:40 smithi047 ceph-mon[29195]: pgmap v53: 97 pgs: 97 active+clean; 15 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.1 MiB/s wr, 20 op/s 2024-04-25T21:01:42.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:42 smithi186 ceph-mon[32529]: pgmap v54: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-04-25T21:01:42.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:42 smithi092 ceph-mon[32524]: pgmap v54: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-04-25T21:01:42.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:42 smithi047 ceph-mon[29195]: pgmap v54: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-04-25T21:01:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:44 smithi186 ceph-mon[32529]: pgmap v55: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-25T21:01:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:44 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:01:44.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:44 smithi092 ceph-mon[32524]: pgmap v55: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-25T21:01:44.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:44 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:01:44.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:44 smithi047 ceph-mon[29195]: pgmap v55: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-25T21:01:44.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:44 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:01:46.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:46 smithi186 ceph-mon[32529]: pgmap v56: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.2 MiB/s wr, 40 op/s 2024-04-25T21:01:46.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:46 smithi092 ceph-mon[32524]: pgmap v56: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.2 MiB/s wr, 40 op/s 2024-04-25T21:01:46.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:46 smithi047 ceph-mon[29195]: pgmap v56: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.2 MiB/s wr, 40 op/s 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 -- 172.21.15.47:0/2175907236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1061b0 msgr2=0x7f5cec10dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 --2- 172.21.15.47:0/2175907236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1061b0 0x7f5cec10dd80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009ca0 tx=0x7f5ce402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 -- 172.21.15.47:0/2175907236 shutdown_connections 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 --2- 172.21.15.47:0/2175907236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1061b0 0x7f5cec10dd80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 --2- 172.21.15.47:0/2175907236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec101a90 0x7f5cec105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 --2- 172.21.15.47:0/2175907236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cec1010e0 0x7f5cec1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 -- 172.21.15.47:0/2175907236 >> 172.21.15.47:0/2175907236 conn(0x7f5cec0771d0 msgr2=0x7f5cec0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 -- 172.21.15.47:0/2175907236 shutdown_connections 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 -- 172.21.15.47:0/2175907236 wait complete. 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.010+0000 7f5cf1491640 1 Processor -- start 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5cf1491640 1 -- start start 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5cf1491640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1010e0 0x7f5cec10e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5cf1491640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec1061b0 0x7f5cec10ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5cf1491640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cec10f190 0x7f5cec1a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5cf1491640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec112230 con 0x7f5cec1010e0 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5cf1491640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1120b0 con 0x7f5cec10f190 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5cf1491640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1123b0 con 0x7f5cec1061b0 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1010e0 0x7f5cec10e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1010e0 0x7f5cec10e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60612/0 (socket says 172.21.15.47:60612) 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceaffd640 1 -- 172.21.15.47:0/3844434620 learned_addr learned my addr 172.21.15.47:0/3844434620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cec10f190 0x7f5cec1a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec1061b0 0x7f5cec10ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceaffd640 1 -- 172.21.15.47:0/3844434620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec1061b0 msgr2=0x7f5cec10ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec1061b0 0x7f5cec10ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceaffd640 1 -- 172.21.15.47:0/3844434620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cec10f190 msgr2=0x7f5cec1a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cec10f190 0x7f5cec1a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceaffd640 1 -- 172.21.15.47:0/3844434620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec1a6f00 con 0x7f5cec1010e0 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.011+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cec10f190 0x7f5cec1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.012+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1010e0 0x7f5cec10e710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00d950 tx=0x7f5cdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.012+0000 7f5cd3fff640 1 -- 172.21.15.47:0/3844434620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc017030 con 0x7f5cec1010e0 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.012+0000 7f5cd3fff640 1 -- 172.21.15.47:0/3844434620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc004020 con 0x7f5cec1010e0 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.012+0000 7f5cf1491640 1 -- 172.21.15.47:0/3844434620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec1a71f0 con 0x7f5cec1010e0 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.012+0000 7f5cd3fff640 1 -- 172.21.15.47:0/3844434620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0047a0 con 0x7f5cec1010e0 2024-04-25T21:01:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.012+0000 7f5cf1491640 1 -- 172.21.15.47:0/3844434620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec1a7790 con 0x7f5cec1010e0 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.014+0000 7f5cd3fff640 1 -- 172.21.15.47:0/3844434620 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5cdc009a30 con 0x7f5cec1010e0 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.014+0000 7f5cf41a9740 1 Processor -- start 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.014+0000 7f5cf41a9740 1 -- start start 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1620 0x55c4b2cb1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cbc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cbebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2c16970 con 0x55c4b2cb1f40 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2bb34a0 con 0x55c4b2cb1620 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2c12930 con 0x55c4b2cbc800 2024-04-25T21:01:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cbc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cd3fff640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5cd40a3e40 0x7f5cd40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cbc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60618/0 (socket says 172.21.15.47:60618) 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 -- 172.21.15.47:0/3737496870 learned_addr learned my addr 172.21.15.47:0/3737496870 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1620 0x55c4b2cb1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cbebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5cd40a3e40 0x7f5cd40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cd3fff640 1 -- 172.21.15.47:0/3844434620 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f5cdc0d02f0 con 0x7f5cec1010e0 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 -- 172.21.15.47:0/3737496870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 msgr2=0x55c4b2cbebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cbebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 -- 172.21.15.47:0/3737496870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1620 msgr2=0x55c4b2cb1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1620 0x55c4b2cb1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 -- 172.21.15.47:0/3737496870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b2929210 con 0x55c4b2cb1f40 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cbebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cbc2c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc09bf40 tx=0x7f5cdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c244dc75677161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.015+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5cd40a3e40 0x7f5cd40a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4004af0 tx=0x7f5ce40095c0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cd17fa640 1 -- 172.21.15.47:0/3737496870 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc09c240 con 0x55c4b2cb1f40 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cd17fa640 1 -- 172.21.15.47:0/3737496870 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc0a05f0 con 0x55c4b2cb1f40 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cd17fa640 1 -- 172.21.15.47:0/3737496870 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0cdc70 con 0x55c4b2cb1f40 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 -- 172.21.15.47:0/3737496870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 msgr2=0x55c4b2cbc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cbc2c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc09bf40 tx=0x7f5cdc000bc0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 -- 172.21.15.47:0/3737496870 shutdown_connections 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cbebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cbc2c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/3737496870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1620 0x55c4b2cb1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 -- 172.21.15.47:0/3737496870 >> 172.21.15.47:0/3737496870 conn(0x55c4b2cadb30 msgr2=0x55c4b2cadf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 -- 172.21.15.47:0/3737496870 shutdown_connections 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.016+0000 7f5cf41a9740 1 -- 172.21.15.47:0/3737496870 wait complete. 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 1 -- start start 2024-04-25T21:01:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 1 Processor -- start 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cbc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cb9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cb99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2cbf370 con 0x55c4b2cb1620 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2bb10b0 con 0x55c4b2cb1f40 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2cbf4f0 con 0x55c4b2cbc800 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cf41a9740 10 client.0 _ll_register_callbacks cb 0x55c4b2b9ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cbc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cbc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60620/0 (socket says 172.21.15.47:60620) 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceaffd640 1 -- 172.21.15.47:0/746507575 learned_addr learned my addr 172.21.15.47:0/746507575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cb99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cb9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceaffd640 1 -- 172.21.15.47:0/746507575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 msgr2=0x55c4b2cb99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cb99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceaffd640 1 -- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 msgr2=0x55c4b2cb9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cb9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5ceaffd640 1 -- 172.21.15.47:0/746507575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b29f5620 con 0x55c4b2cb1620 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.017+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cb9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cbc170 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc004770 tx=0x7f5cdc009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55332badd05d6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cceff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cceff5640 1 -- 172.21.15.47:0/746507575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0041d0 con 0x55c4b2cb1620 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cceff5640 1 -- 172.21.15.47:0/746507575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f5cdc0cdc70 con 0x55c4b2cb1620 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 1 -- 172.21.15.47:0/746507575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cdc0a05f0 con 0x55c4b2cb1620 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cceff5640 1 -- 172.21.15.47:0/746507575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cdc09c240 con 0x55c4b2cb1620 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 10 client.15543 fetch_fsmap learned FSMap version 28 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 1 -- 172.21.15.47:0/746507575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4b2ccb2a0 con 0x55c4b2cb1620 2024-04-25T21:01:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cceff5640 1 -- 172.21.15.47:0/746507575 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc0a05f0 con 0x55c4b2cb1620 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cceff5640 1 -- 172.21.15.47:0/746507575 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f5cdc0b0ce0 con 0x55c4b2cb1620 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 10 client.15543 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 20 client.15543 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c4b2ccc5f0 0x55c4b2cce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 4 client.15543 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.018+0000 7f5cf41a9740 1 -- 172.21.15.47:0/746507575 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4b2c24390 con 0x55c4b2ccc5f0 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.019+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c4b2ccc5f0 0x55c4b2cce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.019+0000 7f5cceff5640 10 client.15543 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.019+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c4b2ccc5f0 0x55c4b2cce9e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9310ea2c2abad8d server_cookie=bfc9a20d1e389252 in_seq=0 out_seq=0 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cceff5640 1 -- 172.21.15.47:0/746507575 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4b2c24390 con 0x55c4b2ccc5f0 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cceff5640 10 client.15543 handle_command_reply: tid=0 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 client.15543 shutdown 2024-04-25T21:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 -- 172.21.15.47:0/746507575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 msgr2=0x55c4b2cbc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cbc170 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc004770 tx=0x7f5cdc009f50 comp rx=0 tx=0).stop 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 -- 172.21.15.47:0/746507575 shutdown_connections 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c4b2ccc5f0 0x55c4b2cce9e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cb99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cb9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/746507575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cbc170 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cceff5640 0 client.15543 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 -- 172.21.15.47:0/746507575 >> 172.21.15.47:0/746507575 conn(0x55c4b2cadb30 msgr2=0x55c4b2cb38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 -- 172.21.15.47:0/746507575 shutdown_connections 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 1 -- 172.21.15.47:0/746507575 wait complete. 2024-04-25T21:01:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.020+0000 7f5cf41a9740 20 client.15543 trim_cache size 0 max 16384 2024-04-25T21:01:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5cf41a9740 1 Processor -- start 2024-04-25T21:01:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5cf41a9740 1 -- start start 2024-04-25T21:01:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cd40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cd4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1b25e0 con 0x55c4b2cb1620 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1afeb0 con 0x55c4b2cb1f40 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1add20 con 0x55c4b2cbc800 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60632/0 (socket says 172.21.15.47:60632) 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.035+0000 7f5ceaffd640 1 -- 172.21.15.47:0/323053316 learned_addr learned my addr 172.21.15.47:0/323053316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cd40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cd4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5ceaffd640 1 -- 172.21.15.47:0/323053316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 msgr2=0x55c4b2cd4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cd4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5ceaffd640 1 -- 172.21.15.47:0/323053316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 msgr2=0x55c4b2cd40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cd40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5ceaffd640 1 -- 172.21.15.47:0/323053316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b2cb8410 con 0x55c4b2cb1620 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cd4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc8230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc09c400 tx=0x7f5cdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4eda0f3c60fd740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5cd0ff9640 1 -- 172.21.15.47:0/323053316 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc097f90 con 0x55c4b2cb1620 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5cd0ff9640 1 -- 172.21.15.47:0/323053316 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc0a05f0 con 0x55c4b2cb1620 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.036+0000 7f5cd0ff9640 1 -- 172.21.15.47:0/323053316 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0a4a20 con 0x55c4b2cb1620 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- 172.21.15.47:0/323053316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 msgr2=0x55c4b2cc8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc8230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc09c400 tx=0x7f5cdc000bc0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- 172.21.15.47:0/323053316 shutdown_connections 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cbc800 0x55c4b2cd4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cd40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/323053316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc8230 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- 172.21.15.47:0/323053316 >> 172.21.15.47:0/323053316 conn(0x55c4b2cadb30 msgr2=0x55c4b2cac420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- 172.21.15.47:0/323053316 shutdown_connections 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- 172.21.15.47:0/323053316 wait complete. 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- start start 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 Processor -- start 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cc1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cbc800 0x55c4b2cc25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1ae370 con 0x55c4b2cb1f40 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1af7e0 con 0x55c4b2cbc800 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1aea40 con 0x55c4b2cb1620 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cf41a9740 10 client.0 _ll_register_callbacks cb 0x7f5cdc004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.037+0000 7f5cea7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cc1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cea7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cc1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60644/0 (socket says 172.21.15.47:60644) 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cea7fc640 1 -- 172.21.15.47:0/2457424110 learned_addr learned my addr 172.21.15.47:0/2457424110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:01:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cbc800 0x55c4b2cc25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cea7fc640 1 -- 172.21.15.47:0/2457424110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cb1620 msgr2=0x55c4b2cc4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cea7fc640 1 -- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cbc800 msgr2=0x55c4b2cc25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cbc800 0x55c4b2cc25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cea7fc640 1 -- 172.21.15.47:0/2457424110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b2cb1d70 con 0x55c4b2cb1f40 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5ceb7fe640 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cbc800 0x55c4b2cc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cea7fc640 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cc1fb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005b60 tx=0x7f5ce000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29f76891bc3129a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cceff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cceff5640 1 -- 172.21.15.47:0/2457424110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0004440 con 0x55c4b2cb1f40 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cceff5640 1 -- 172.21.15.47:0/2457424110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f5ce0012920 con 0x55c4b2cb1f40 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cf41a9740 1 -- 172.21.15.47:0/2457424110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cdc0a05f0 con 0x55c4b2cb1f40 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cceff5640 1 -- 172.21.15.47:0/2457424110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ce002b050 con 0x55c4b2cb1f40 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cf41a9740 10 client.15549 fetch_fsmap learned FSMap version 28 2024-04-25T21:01:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.038+0000 7f5cf41a9740 1 -- 172.21.15.47:0/2457424110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4b2d328d0 con 0x55c4b2cb1f40 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5cceff5640 1 -- 172.21.15.47:0/2457424110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc0a05f0 con 0x55c4b2cb1f40 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5cceff5640 1 -- 172.21.15.47:0/2457424110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f5ce000f040 con 0x55c4b2cb1f40 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5cf41a9740 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5cf41a9740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5cf41a9740 20 client.15549 populate_metadata read hostname 'smithi047' 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c4b2d33090 0x55c4b2d354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5cf41a9740 4 client.15549 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5cf41a9740 1 -- 172.21.15.47:0/2457424110 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4b2c1e200 con 0x55c4b2d33090 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.039+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c4b2d33090 0x55c4b2d354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.040+0000 7f5cceff5640 10 client.15549 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:01:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:01:47.040+0000 7f5ceaffd640 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c4b2d33090 0x55c4b2d354b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0caaf339241660b server_cookie=c63b119f071ad0c9 in_seq=0 out_seq=0 2024-04-25T21:01:48.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:48 smithi186 ceph-mon[32529]: pgmap v57: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 22 op/s 2024-04-25T21:01:48.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:48 smithi092 ceph-mon[32524]: pgmap v57: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 22 op/s 2024-04-25T21:01:48.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:48 smithi047 ceph-mon[29195]: pgmap v57: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 22 op/s 2024-04-25T21:01:50.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:50 smithi047 ceph-mon[29195]: pgmap v58: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 22 op/s 2024-04-25T21:01:50.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:50 smithi186 ceph-mon[32529]: pgmap v58: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 22 op/s 2024-04-25T21:01:50.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:50 smithi092 ceph-mon[32524]: pgmap v58: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 22 op/s 2024-04-25T21:01:52.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:52 smithi186 ceph-mon[32529]: pgmap v59: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 128 KiB/s wr, 70 op/s 2024-04-25T21:01:52.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:52 smithi092 ceph-mon[32524]: pgmap v59: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 128 KiB/s wr, 70 op/s 2024-04-25T21:01:52.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:52 smithi047 ceph-mon[29195]: pgmap v59: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 128 KiB/s wr, 70 op/s 2024-04-25T21:01:54.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:54 smithi186 ceph-mon[32529]: pgmap v60: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 22 KiB/s wr, 52 op/s 2024-04-25T21:01:54.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:54 smithi092 ceph-mon[32524]: pgmap v60: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 22 KiB/s wr, 52 op/s 2024-04-25T21:01:54.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:54 smithi047 ceph-mon[29195]: pgmap v60: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 22 KiB/s wr, 52 op/s 2024-04-25T21:01:56.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:56 smithi186 ceph-mon[32529]: pgmap v61: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 55 op/s 2024-04-25T21:01:56.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:56 smithi092 ceph-mon[32524]: pgmap v61: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 55 op/s 2024-04-25T21:01:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:56 smithi047 ceph-mon[29195]: pgmap v61: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 55 op/s 2024-04-25T21:01:58.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:01:58 smithi186 ceph-mon[32529]: pgmap v62: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 19 KiB/s wr, 50 op/s 2024-04-25T21:01:58.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:01:58 smithi092 ceph-mon[32524]: pgmap v62: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 19 KiB/s wr, 50 op/s 2024-04-25T21:01:58.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:01:58 smithi047 ceph-mon[29195]: pgmap v62: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 19 KiB/s wr, 50 op/s 2024-04-25T21:02:00.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:00 smithi186 ceph-mon[32529]: pgmap v63: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 19 KiB/s wr, 50 op/s 2024-04-25T21:02:00.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:00 smithi092 ceph-mon[32524]: pgmap v63: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 19 KiB/s wr, 50 op/s 2024-04-25T21:02:00.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:00 smithi047 ceph-mon[29195]: pgmap v63: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 19 KiB/s wr, 50 op/s 2024-04-25T21:02:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:02 smithi186 ceph-mon[32529]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 19 KiB/s wr, 53 op/s 2024-04-25T21:02:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:02 smithi092 ceph-mon[32524]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 19 KiB/s wr, 53 op/s 2024-04-25T21:02:02.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:02 smithi047 ceph-mon[29195]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 19 KiB/s wr, 53 op/s 2024-04-25T21:02:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:04 smithi186 ceph-mon[32529]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:04.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:04 smithi092 ceph-mon[32524]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:04.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:04 smithi047 ceph-mon[29195]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:06.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:06 smithi186 ceph-mon[32529]: pgmap v66: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 233 KiB/s wr, 53 op/s 2024-04-25T21:02:06.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:06 smithi092 ceph-mon[32524]: pgmap v66: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 233 KiB/s wr, 53 op/s 2024-04-25T21:02:06.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:06 smithi047 ceph-mon[29195]: pgmap v66: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 233 KiB/s wr, 53 op/s 2024-04-25T21:02:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:07 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:07 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:07 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:07 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:02:07.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:07 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:07.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:07 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:07.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:07 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:07.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:07 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:02:07.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:07 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:07.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:07 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:07.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:07 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:07.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:07 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:02:08.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:08 smithi186 ceph-mon[32529]: pgmap v67: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 KiB/s wr, 50 op/s 2024-04-25T21:02:08.511 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:08 smithi047 ceph-mon[29195]: pgmap v67: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 KiB/s wr, 50 op/s 2024-04-25T21:02:08.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:08 smithi092 ceph-mon[32524]: pgmap v67: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 KiB/s wr, 50 op/s 2024-04-25T21:02:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:10 smithi186 ceph-mon[32529]: pgmap v68: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 KiB/s wr, 50 op/s 2024-04-25T21:02:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:10 smithi092 ceph-mon[32524]: pgmap v68: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 KiB/s wr, 50 op/s 2024-04-25T21:02:10.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:10 smithi047 ceph-mon[29195]: pgmap v68: 97 pgs: 97 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 KiB/s wr, 50 op/s 2024-04-25T21:02:12.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:12 smithi186 ceph-mon[32529]: pgmap v69: 97 pgs: 97 active+clean; 18 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 278 KiB/s wr, 54 op/s 2024-04-25T21:02:12.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:12 smithi092 ceph-mon[32524]: pgmap v69: 97 pgs: 97 active+clean; 18 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 278 KiB/s wr, 54 op/s 2024-04-25T21:02:12.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:12 smithi047 ceph-mon[29195]: pgmap v69: 97 pgs: 97 active+clean; 18 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 278 KiB/s wr, 54 op/s 2024-04-25T21:02:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:14 smithi186 ceph-mon[32529]: pgmap v70: 97 pgs: 97 active+clean; 18 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 278 KiB/s wr, 51 op/s 2024-04-25T21:02:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:14 smithi092 ceph-mon[32524]: pgmap v70: 97 pgs: 97 active+clean; 18 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 278 KiB/s wr, 51 op/s 2024-04-25T21:02:14.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:14 smithi047 ceph-mon[29195]: pgmap v70: 97 pgs: 97 active+clean; 18 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 278 KiB/s wr, 51 op/s 2024-04-25T21:02:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:16 smithi186 ceph-mon[32529]: pgmap v71: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 281 KiB/s wr, 55 op/s 2024-04-25T21:02:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:16 smithi092 ceph-mon[32524]: pgmap v71: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 281 KiB/s wr, 55 op/s 2024-04-25T21:02:16.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:16 smithi047 ceph-mon[29195]: pgmap v71: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 281 KiB/s wr, 55 op/s 2024-04-25T21:02:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6a777fe700 1 -- 172.21.15.47:0/2636197440 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: -115 ) ==== 8+0+592 (crc 0 0 0) 0x56076c6196b0 con 0x56076c741280 2024-04-25T21:02:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6a777fe700 10 client.15531 handle_command_reply: tid=0 2024-04-25T21:02:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 client.15531 shutdown 2024-04-25T21:02:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/2636197440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6c9f00 msgr2=0x56076c6d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6c9f00 0x56076c6d0710 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6a900cd940 tx=0x7f6a9009b9f0 comp rx=0 tx=0).stop 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/2636197440 shutdown_connections 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56076c741280 0x56076c743670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c6c9f00 0x56076c6d0710 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56076c6be6e0 0x56076c6d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 --2- 172.21.15.47:0/2636197440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56076c6bf000 0x56076c6d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6a777fe700 0 client.15531 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/2636197440 >> 172.21.15.47:0/2636197440 conn(0x56076c5eba20 msgr2=0x56076c6d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/2636197440 shutdown_connections 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 1 -- 172.21.15.47:0/2636197440 wait complete. 2024-04-25T21:02:17.224 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 28, 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "leader": 24488, 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "set_version": 5, 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "sets": { 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "9ebddfb8": { 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 5, 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "age_ref": 0.0, 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "state": { 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "QUIESCING", 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 42.0 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "timeout": 60.0, 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "expiration": 86.2, 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "members": { 2024-04-25T21:02:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "file:/": { 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "excluded": false, 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "state": { 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "QUIESCING", 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 42.0 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:02:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:17.222+0000 7f6aa8cdab80 20 client.15531 trim_cache size 0 max 16384 2024-04-25T21:02:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:Error EINPROGRESS: 2024-04-25T21:02:18.017 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-04-25T21:02:18.018 WARNING:tasks.quiescer.fs.[cephfs]:Set '9ebddfb8' hasn't quiesced after 48.1 seconds (timeout: 60). Dumping ops with locks from all ranks. 2024-04-25T21:02:18.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:02:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:02:18.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:18 smithi047 ceph-mon[29195]: pgmap v72: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 7 op/s 2024-04-25T21:02:18.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:18 smithi186 ceph-mon[32529]: pgmap v72: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 7 op/s 2024-04-25T21:02:18.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:18 smithi092 ceph-mon[32524]: pgmap v72: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 7 op/s 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 -- 172.21.15.47:0/3767598815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968103450 msgr2=0x7f996810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 --2- 172.21.15.47:0/3767598815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968103450 0x7f996810f8b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f996000b0a0 tx=0x7f996000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 -- 172.21.15.47:0/3767598815 shutdown_connections 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 --2- 172.21.15.47:0/3767598815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968103450 0x7f996810f8b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 --2- 172.21.15.47:0/3767598815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9968108a50 0x7f9968108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 --2- 172.21.15.47:0/3767598815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9968102a90 0x7f9968102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 -- 172.21.15.47:0/3767598815 >> 172.21.15.47:0/3767598815 conn(0x7f9968079ba0 msgr2=0x7f9968079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 -- 172.21.15.47:0/3767598815 shutdown_connections 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.949+0000 7f996fc80700 1 -- 172.21.15.47:0/3767598815 wait complete. 2024-04-25T21:02:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996fc80700 1 Processor -- start 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996fc80700 1 -- start start 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996fc80700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9968102a90 0x7f99681990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996fc80700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9968103450 0x7f99681995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996fc80700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968108a50 0x7f996819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996fc80700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9968112be0 con 0x7f9968108a50 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996fc80700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9968112a60 con 0x7f9968102a90 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996fc80700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9968112d60 con 0x7f9968103450 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996e21d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968108a50 0x7f996819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996e21d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968108a50 0x7f996819d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45210/0 (socket says 172.21.15.47:45210) 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996e21d700 1 -- 172.21.15.47:0/2620924616 learned_addr learned my addr 172.21.15.47:0/2620924616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996da1c700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9968102a90 0x7f99681990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996d21b700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9968103450 0x7f99681995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996e21d700 1 -- 172.21.15.47:0/2620924616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9968103450 msgr2=0x7f99681995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996e21d700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9968103450 0x7f99681995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996e21d700 1 -- 172.21.15.47:0/2620924616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9968102a90 msgr2=0x7f99681990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996e21d700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9968102a90 0x7f99681990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996e21d700 1 -- 172.21.15.47:0/2620924616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996819e0c0 con 0x7f9968108a50 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.950+0000 7f996d21b700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9968103450 0x7f99681995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.951+0000 7f996e21d700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968108a50 0x7f996819d980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f996000b830 tx=0x7f9960004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.951+0000 7f995effd700 1 -- 172.21.15.47:0/2620924616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960021070 con 0x7f9968108a50 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.951+0000 7f995effd700 1 -- 172.21.15.47:0/2620924616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9960007e30 con 0x7f9968108a50 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.951+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996819e350 con 0x7f9968108a50 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.951+0000 7f995effd700 1 -- 172.21.15.47:0/2620924616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600047d0 con 0x7f9968108a50 2024-04-25T21:02:19.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.951+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99681a5c30 con 0x7f9968108a50 2024-04-25T21:02:19.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.952+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9968069730 con 0x7f9968108a50 2024-04-25T21:02:19.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.953+0000 7f995effd700 1 -- 172.21.15.47:0/2620924616 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9960007980 con 0x7f9968108a50 2024-04-25T21:02:19.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.954+0000 7f995effd700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f99540a3e30 0x7f99540a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:19.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.954+0000 7f996da1c700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f99540a3e30 0x7f99540a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:19.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.954+0000 7f995effd700 1 -- 172.21.15.47:0/2620924616 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f996000f020 con 0x7f9968108a50 2024-04-25T21:02:19.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.954+0000 7f996da1c700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f99540a3e30 0x7f99540a62f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9964005950 tx=0x7f996400a300 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:19.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:19.958+0000 7f995effd700 1 -- 172.21.15.47:0/2620924616 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f996002a020 con 0x7f9968108a50 2024-04-25T21:02:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.219+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f996802df80 con 0x7f9968108a50 2024-04-25T21:02:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.219+0000 7f995effd700 1 -- 172.21.15.47:0/2620924616 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11856 (secure 0 0 0) 0x7f996009e430 con 0x7f9968108a50 2024-04-25T21:02:20.220 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:02:20.220 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T21:01:42.311105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:02:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f99540a3e30 msgr2=0x7f99540a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f99540a3e30 0x7f99540a62f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9964005950 tx=0x7f996400a300 comp rx=0 tx=0).stop 2024-04-25T21:02:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968108a50 msgr2=0x7f996819d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968108a50 0x7f996819d980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f996000b830 tx=0x7f9960004ce0 comp rx=0 tx=0).stop 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 shutdown_connections 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f99540a3e30 0x7f99540a62f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9968108a50 0x7f996819d980 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9968102a90 0x7f99681990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 --2- 172.21.15.47:0/2620924616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9968103450 0x7f99681995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 >> 172.21.15.47:0/2620924616 conn(0x7f9968079ba0 msgr2=0x7f9968101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 shutdown_connections 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:20.222+0000 7f996fc80700 1 -- 172.21.15.47:0/2620924616 wait complete. 2024-04-25T21:02:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-04-25T21:02:20.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:20 smithi047 ceph-mon[29195]: pgmap v73: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 7 op/s 2024-04-25T21:02:20.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:20 smithi186 ceph-mon[32529]: pgmap v73: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 7 op/s 2024-04-25T21:02:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:20 smithi092 ceph-mon[32524]: pgmap v73: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 7 op/s 2024-04-25T21:02:20.833 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (e) to a remote file /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.e.json 2024-04-25T21:02:20.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.1:2 ops --flags=locks --path=/var/run/ceph/ops-9ebddfb8-1-mds.e.json 2024-04-25T21:02:21.092 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:02:21.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:21 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2620924616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:02:21.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:21 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2620924616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:02:21.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:21 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2620924616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:02:22.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:22 smithi047 ceph-mon[29195]: pgmap v74: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-25T21:02:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:22 smithi092 ceph-mon[32524]: pgmap v74: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-25T21:02:22.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:22 smithi186 ceph-mon[32529]: pgmap v74: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-25T21:02:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 -- 172.21.15.47:0/361961535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 msgr2=0x7f6eb4102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/361961535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 0x7f6eb4102f40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009990 tx=0x7f6ea8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 -- 172.21.15.47:0/361961535 shutdown_connections 2024-04-25T21:02:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/361961535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb4103480 0x7f6eb410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/361961535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb4108a70 0x7f6eb4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/361961535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 0x7f6eb4102f40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 -- 172.21.15.47:0/361961535 >> 172.21.15.47:0/361961535 conn(0x7f6eb4079bb0 msgr2=0x7f6eb4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 -- 172.21.15.47:0/361961535 shutdown_connections 2024-04-25T21:02:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.877+0000 7f6eb8de0700 1 -- 172.21.15.47:0/361961535 wait complete. 2024-04-25T21:02:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb8de0700 1 Processor -- start 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb8de0700 1 -- start start 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb8de0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 0x7f6eb4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb8de0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb4103480 0x7f6eb4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb8de0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb4108a70 0x7f6eb419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb8de0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4112c00 con 0x7f6eb4102ac0 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb8de0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4112a80 con 0x7f6eb4103480 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb8de0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4112d80 con 0x7f6eb4108a70 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 0x7f6eb4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 0x7f6eb4199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45226/0 (socket says 172.21.15.47:45226) 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb259c700 1 -- 172.21.15.47:0/1020990818 learned_addr learned my addr 172.21.15.47:0/1020990818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb2d9d700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb4108a70 0x7f6eb419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb4103480 0x7f6eb4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb259c700 1 -- 172.21.15.47:0/1020990818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb4108a70 msgr2=0x7f6eb419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb259c700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb4108a70 0x7f6eb419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb259c700 1 -- 172.21.15.47:0/1020990818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb4103480 msgr2=0x7f6eb4199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb259c700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb4103480 0x7f6eb4199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.878+0000 7f6eb259c700 1 -- 172.21.15.47:0/1020990818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb419e0d0 con 0x7f6eb4102ac0 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.879+0000 7f6eb259c700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 0x7f6eb4199110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400d950 tx=0x7f6ea400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.879+0000 7f6ea37fe700 1 -- 172.21.15.47:0/1020990818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4016070 con 0x7f6eb4102ac0 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.879+0000 7f6eb8de0700 1 -- 172.21.15.47:0/1020990818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb419e3c0 con 0x7f6eb4102ac0 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.879+0000 7f6ea37fe700 1 -- 172.21.15.47:0/1020990818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea4009df0 con 0x7f6eb4102ac0 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.879+0000 7f6ea37fe700 1 -- 172.21.15.47:0/1020990818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400f3c0 con 0x7f6eb4102ac0 2024-04-25T21:02:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.879+0000 7f6eb8de0700 1 -- 172.21.15.47:0/1020990818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb41a5c40 con 0x7f6eb4102ac0 2024-04-25T21:02:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.881+0000 7f6ea37fe700 1 -- 172.21.15.47:0/1020990818 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6ea4019070 con 0x7f6eb4102ac0 2024-04-25T21:02:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.881+0000 7f6ea37fe700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6e9c0a3e30 0x7f6e9c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.882+0000 7f6ea37fe700 1 -- 172.21.15.47:0/1020990818 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6ea40d0360 con 0x7f6eb4102ac0 2024-04-25T21:02:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.882+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6e9c0a3e30 0x7f6e9c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.882+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6e9c0a3e30 0x7f6e9c0a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8006010 tx=0x7f6ea801a040 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.882+0000 7f6ec3669b80 1 Processor -- start 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.882+0000 7f6ec3669b80 1 -- start start 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f92a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 0x560a01f9d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 0x560a01fa0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560a01ee5d70 con 0x560a01f92630 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560a01e8ca10 con 0x560a01f9de50 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560a01ee1d30 con 0x560a01f92f50 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f92a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f92a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45232/0 (socket says 172.21.15.47:45232) 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 -- 172.21.15.47:0/3463661418 learned_addr learned my addr 172.21.15.47:0/3463661418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 0x560a01f9d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb2d9d700 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 0x560a01fa0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 -- 172.21.15.47:0/3463661418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 msgr2=0x560a01f9d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 0x560a01f9d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 -- 172.21.15.47:0/3463661418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 msgr2=0x560a01fa0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 0x560a01fa0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 -- 172.21.15.47:0/3463661418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a01e50820 con 0x560a01f92630 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6eb259c700 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f92a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4098c80 tx=0x7f6ea409c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a7a8a6c5c74ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.883+0000 7f6ea0ff9700 1 -- 172.21.15.47:0/3463661418 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4020040 con 0x560a01f92630 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ea0ff9700 1 -- 172.21.15.47:0/3463661418 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea40a4f20 con 0x560a01f92630 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ea0ff9700 1 -- 172.21.15.47:0/3463661418 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40a4a90 con 0x560a01f92630 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3463661418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 msgr2=0x560a01f92a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f92a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4098c80 tx=0x7f6ea409c2b0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3463661418 shutdown_connections 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f92a10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 0x560a01fa0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3463661418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 0x560a01f9d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3463661418 >> 172.21.15.47:0/3463661418 conn(0x560a01ebf970 msgr2=0x560a01f8eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3463661418 shutdown_connections 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.884+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3463661418 wait complete. 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 1 -- start start 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 1 Processor -- start 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f9d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92f50 0x560a01f9a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 0x560a01f9aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560a01fa0880 con 0x560a01f92630 2024-04-25T21:02:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560a01e9f690 con 0x560a01f92f50 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560a01fa0a00 con 0x560a01f9de50 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f9d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6ec3669b80 10 client.0 _ll_register_callbacks cb 0x560a01e7ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f9d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45242/0 (socket says 172.21.15.47:45242) 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6eb259c700 1 -- 172.21.15.47:0/882051254 learned_addr learned my addr 172.21.15.47:0/882051254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.885+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92f50 0x560a01f9a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6eb2d9d700 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 0x560a01f9aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6eb259c700 1 -- 172.21.15.47:0/882051254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 msgr2=0x560a01f9aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6eb259c700 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 0x560a01f9aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6eb259c700 1 -- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92f50 msgr2=0x560a01f9a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6eb259c700 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92f50 0x560a01f9a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6eb259c700 1 -- 172.21.15.47:0/882051254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a01e339b0 con 0x560a01f92630 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92f50 0x560a01f9a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6eb259c700 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f9d670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400b2f0 tx=0x7f6ea409a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f109dcb6b8cef9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6e99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6e99ffb700 1 -- 172.21.15.47:0/882051254 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea407d020 con 0x560a01f92630 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6e99ffb700 1 -- 172.21.15.47:0/882051254 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6ea409c710 con 0x560a01f92630 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6ec3669b80 1 -- 172.21.15.47:0/882051254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea40a4f20 con 0x560a01f92630 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6e99ffb700 1 -- 172.21.15.47:0/882051254 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ea40a6070 con 0x560a01f92630 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6ec3669b80 10 client.15567 fetch_fsmap learned FSMap version 28 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6ec3669b80 1 -- 172.21.15.47:0/882051254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a01fab1d0 con 0x560a01f92630 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.886+0000 7f6e99ffb700 1 -- 172.21.15.47:0/882051254 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea40a4f20 con 0x560a01f92630 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6e99ffb700 1 -- 172.21.15.47:0/882051254 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f6ea409b700 con 0x560a01f92630 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6ec3669b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6ec3669b80 10 client.15567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6ec3669b80 20 client.15567 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x560a01fad6a0 0x560a01fafa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6ec3669b80 4 client.15567 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6ec3669b80 1 -- 172.21.15.47:0/882051254 --> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a01ef3560 con 0x560a01fad6a0 2024-04-25T21:02:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x560a01fad6a0 0x560a01fafa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x560a01fad6a0 0x560a01fafa90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fbc3c5fdf50f2bb server_cookie=54cf7f00b1f996e2 in_seq=0 out_seq=0 2024-04-25T21:02:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.887+0000 7f6e99ffb700 10 client.15567 ms_handle_connect on v2:172.21.15.92:6834/2097108178 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.888+0000 7f6e99ffb700 1 -- 172.21.15.47:0/882051254 <== mds.2 v2:172.21.15.92:6834/2097108178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a01ef3560 con 0x560a01fad6a0 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.888+0000 7f6e99ffb700 10 client.15567 handle_command_reply: tid=0 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.888+0000 7f6ec3669b80 1 client.15567 shutdown 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 -- 172.21.15.47:0/882051254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 msgr2=0x560a01f9d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f9d670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400b2f0 tx=0x7f6ea409a5b0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 -- 172.21.15.47:0/882051254 shutdown_connections 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x560a01fad6a0 0x560a01fafa90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f9d670 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 0x560a01f9aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/882051254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92f50 0x560a01f9a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 -- 172.21.15.47:0/882051254 >> 172.21.15.47:0/882051254 conn(0x560a01ebf970 msgr2=0x560a01f94fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6e99ffb700 0 client.15567 ms_handle_reset on v2:172.21.15.92:6834/2097108178 2024-04-25T21:02:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 -- 172.21.15.47:0/882051254 shutdown_connections 2024-04-25T21:02:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 1 -- 172.21.15.47:0/882051254 wait complete. 2024-04-25T21:02:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.889+0000 7f6ec3669b80 20 client.15567 trim_cache size 0 max 16384 2024-04-25T21:02:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6ec3669b80 1 Processor -- start 2024-04-25T21:02:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6ec3669b80 1 -- start start 2024-04-25T21:02:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f99b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 0x560a01f9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 0x560a01fb26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41ad9b0 con 0x560a01f92630 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41ab240 con 0x560a01f9de50 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41a90b0 con 0x560a01f92f50 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f99b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f99b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45256/0 (socket says 172.21.15.47:45256) 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6eb259c700 1 -- 172.21.15.47:0/843378204 learned_addr learned my addr 172.21.15.47:0/843378204 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.907+0000 7f6eb2d9d700 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 0x560a01fb26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 0x560a01f9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6eb259c700 1 -- 172.21.15.47:0/843378204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 msgr2=0x560a01f9d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6eb259c700 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 0x560a01f9d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6eb259c700 1 -- 172.21.15.47:0/843378204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 msgr2=0x560a01fb26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6eb259c700 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 0x560a01fb26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6eb259c700 1 -- 172.21.15.47:0/843378204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a01f98f00 con 0x560a01f92630 2024-04-25T21:02:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6eb259c700 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f99b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40d02d0 tx=0x7f6ea40bb380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6db9b981c1c7a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6e9bfff700 1 -- 172.21.15.47:0/843378204 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea407d070 con 0x560a01f92630 2024-04-25T21:02:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6e9bfff700 1 -- 172.21.15.47:0/843378204 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea40a4f20 con 0x560a01f92630 2024-04-25T21:02:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.908+0000 7f6e9bfff700 1 -- 172.21.15.47:0/843378204 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4020040 con 0x560a01f92630 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- 172.21.15.47:0/843378204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 msgr2=0x560a01f99b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f99b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40d02d0 tx=0x7f6ea40bb380 comp rx=0 tx=0).stop 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- 172.21.15.47:0/843378204 shutdown_connections 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92630 0x560a01f99b40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f9de50 0x560a01fb26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/843378204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f92f50 0x560a01f9d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- 172.21.15.47:0/843378204 >> 172.21.15.47:0/843378204 conn(0x560a01ebf970 msgr2=0x560a01f8dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- 172.21.15.47:0/843378204 shutdown_connections 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- 172.21.15.47:0/843378204 wait complete. 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- start start 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 Processor -- start 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92630 0x560a01fb0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92f50 0x560a01fb06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 0x560a01fb0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41a9700 con 0x560a01f92f50 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41aab70 con 0x560a01f92630 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41a9dd0 con 0x560a01f9de50 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.909+0000 7f6ec3669b80 10 client.0 _ll_register_callbacks cb 0x560a01e7ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92f50 0x560a01fb06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92630 0x560a01fb0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92f50 0x560a01fb06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45272/0 (socket says 172.21.15.47:45272) 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 -- 172.21.15.47:0/3887123951 learned_addr learned my addr 172.21.15.47:0/3887123951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb2d9d700 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 0x560a01fb0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 -- 172.21.15.47:0/3887123951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 msgr2=0x560a01fb0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 0x560a01fb0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 -- 172.21.15.47:0/3887123951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92630 msgr2=0x560a01fb0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92630 0x560a01fb0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 -- 172.21.15.47:0/3887123951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a02012900 con 0x560a01f92f50 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6eb1d9b700 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92f50 0x560a01fb06a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009990 tx=0x7f6ea8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6326e61190fb2382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6e99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6e99ffb700 1 -- 172.21.15.47:0/3887123951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800fbd0 con 0x560a01f92f50 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6e99ffb700 1 -- 172.21.15.47:0/3887123951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6ea800fd70 con 0x560a01f92f50 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3887123951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea40a4f20 con 0x560a01f92f50 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6e99ffb700 1 -- 172.21.15.47:0/3887123951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ea8045050 con 0x560a01f92f50 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.910+0000 7f6ec3669b80 10 client.15573 fetch_fsmap learned FSMap version 28 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3887123951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a02013bf0 con 0x560a01f92f50 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6e99ffb700 1 -- 172.21.15.47:0/3887123951 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea40a4f20 con 0x560a01f92f50 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6e99ffb700 1 -- 172.21.15.47:0/3887123951 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f6ea802f3f0 con 0x560a01f92f50 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6ec3669b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6ec3669b80 10 client.15573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6ec3669b80 20 client.15573 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x560a02014380 0x560a020167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6ec3669b80 4 client.15573 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-9ebddfb8-1-mds.e.json"}] 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3887123951 --> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-9ebddfb8-1-mds.e.json"}) -- 0x560a01eed600 con 0x560a02014380 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6eb259c700 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x560a02014380 0x560a020167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6e99ffb700 10 client.15573 ms_handle_connect on v2:172.21.15.92:6834/2097108178 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.911+0000 7f6eb259c700 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x560a02014380 0x560a020167a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38456366c5ed2dcb server_cookie=3872fffa2a3beb1d in_seq=0 out_seq=0 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6e99ffb700 1 -- 172.21.15.47:0/3887123951 <== mds.2 v2:172.21.15.92:6834/2097108178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x560a01eed600 con 0x560a02014380 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6e99ffb700 10 client.15573 handle_command_reply: tid=0 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 client.15573 shutdown 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3887123951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92f50 msgr2=0x560a01fb06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92f50 0x560a01fb06a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009990 tx=0x7f6ea8009310 comp rx=0 tx=0).stop 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3887123951 shutdown_connections 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.92:6834/2097108178,v1:172.21.15.92:6835/2097108178] conn(0x560a02014380 0x560a020167a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a01f9de50 0x560a01fb0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a01f92630 0x560a01fb0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6e99ffb700 0 client.15573 ms_handle_reset on v2:172.21.15.92:6834/2097108178 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 --2- 172.21.15.47:0/3887123951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a01f92f50 0x560a01fb06a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3887123951 >> 172.21.15.47:0/3887123951 conn(0x560a01ebf970 msgr2=0x560a01f944e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3887123951 shutdown_connections 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 1 -- 172.21.15.47:0/3887123951 wait complete. 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6ec3669b80 20 client.15573 trim_cache size 0 max 16384 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/var/run/ceph/ops-9ebddfb8-1-mds.e.json" 2024-04-25T21:02:22.918 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6eb8de0700 1 -- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6e9c0a3e30 msgr2=0x7f6e9c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6e9c0a3e30 0x7f6e9c0a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8006010 tx=0x7f6ea801a040 comp rx=0 tx=0).stop 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6eb8de0700 1 -- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 msgr2=0x7f6eb4199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.913+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 0x7f6eb4199110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400d950 tx=0x7f6ea400dc60 comp rx=0 tx=0).stop 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.914+0000 7f6eb8de0700 1 -- 172.21.15.47:0/1020990818 shutdown_connections 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.914+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6e9c0a3e30 0x7f6e9c0a62f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.914+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6eb4102ac0 0x7f6eb4199110 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.914+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb4108a70 0x7f6eb419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.914+0000 7f6eb8de0700 1 --2- 172.21.15.47:0/1020990818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb4103480 0x7f6eb4199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.914+0000 7f6eb8de0700 1 -- 172.21.15.47:0/1020990818 >> 172.21.15.47:0/1020990818 conn(0x7f6eb4079bb0 msgr2=0x7f6eb41018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.914+0000 7f6eb8de0700 1 -- 172.21.15.47:0/1020990818 shutdown_connections 2024-04-25T21:02:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:22.914+0000 7f6eb8de0700 1 -- 172.21.15.47:0/1020990818 wait complete. 2024-04-25T21:02:23.544 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (h) to a remote file /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.h.json 2024-04-25T21:02:23.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.1:1 ops --flags=locks --path=/var/run/ceph/ops-9ebddfb8-1-mds.h.json 2024-04-25T21:02:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:02:24.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:24 smithi092 ceph-mon[32524]: pgmap v75: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2024-04-25T21:02:24.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:24 smithi047 ceph-mon[29195]: pgmap v75: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2024-04-25T21:02:24.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:24 smithi186 ceph-mon[32529]: pgmap v75: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/1748597290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 msgr2=0x7fb4d0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/1748597290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 0x7fb4d0108e30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009990 tx=0x7fb4c0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/1748597290 shutdown_connections 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/1748597290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 0x7fb4d0108e30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/1748597290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0103450 0x7fb4d010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/1748597290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d0102a90 0x7fb4d0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/1748597290 >> 172.21.15.47:0/1748597290 conn(0x7fb4d0079ba0 msgr2=0x7fb4d0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/1748597290 shutdown_connections 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.519+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/1748597290 wait complete. 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4d4f8e700 1 Processor -- start 2024-04-25T21:02:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4d4f8e700 1 -- start start 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4d4f8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0102a90 0x7fb4d0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4d4f8e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d0103450 0x7fb4d0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4d4f8e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 0x7fb4d00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4d4f8e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0112c00 con 0x7fb4d0108a50 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4d4f8e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0112a80 con 0x7fb4d0102a90 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4d4f8e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0112d80 con 0x7fb4d0103450 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 0x7fb4d00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 0x7fb4d00757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49280/0 (socket says 172.21.15.47:49280) 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4ced9d700 1 -- 172.21.15.47:0/833553107 learned_addr learned my addr 172.21.15.47:0/833553107 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0102a90 0x7fb4d0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.520+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d0103450 0x7fb4d0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4ced9d700 1 -- 172.21.15.47:0/833553107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d0103450 msgr2=0x7fb4d0075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d0103450 0x7fb4d0075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4ced9d700 1 -- 172.21.15.47:0/833553107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0102a90 msgr2=0x7fb4d0074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0102a90 0x7fb4d0074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4ced9d700 1 -- 172.21.15.47:0/833553107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d01a7660 con 0x7fb4d0108a50 2024-04-25T21:02:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0102a90 0x7fb4d0074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 0x7fb4d00757c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8007ee0 tx=0x7fb4c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4bf7fe700 1 -- 172.21.15.47:0/833553107 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800dbd0 con 0x7fb4d0108a50 2024-04-25T21:02:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/833553107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d01a7950 con 0x7fb4d0108a50 2024-04-25T21:02:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4bf7fe700 1 -- 172.21.15.47:0/833553107 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c800dd70 con 0x7fb4d0108a50 2024-04-25T21:02:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4bf7fe700 1 -- 172.21.15.47:0/833553107 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8005990 con 0x7fb4d0108a50 2024-04-25T21:02:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.521+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/833553107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d01a7e90 con 0x7fb4d0108a50 2024-04-25T21:02:25.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.523+0000 7fb4bf7fe700 1 -- 172.21.15.47:0/833553107 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb4c801a020 con 0x7fb4d0108a50 2024-04-25T21:02:25.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.524+0000 7fb4bf7fe700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4b80a3db0 0x7fb4b80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.524+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4b80a3db0 0x7fb4b80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.524+0000 7fb4bf7fe700 1 -- 172.21.15.47:0/833553107 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fb4c80d0000 con 0x7fb4d0108a50 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.524+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4b80a3db0 0x7fb4b80a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0006010 tx=0x7fb4c000b540 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.525+0000 7fb4df817b80 1 Processor -- start 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.525+0000 7fb4df817b80 1 -- start start 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.525+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee66ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.525+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee679910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.525+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee67c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.525+0000 7fb4df817b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dbee5c1d70 con 0x55dbee679e50 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.525+0000 7fb4df817b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dbee568a10 con 0x55dbee66ef50 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.525+0000 7fb4df817b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dbee5bdd30 con 0x55dbee66e630 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee67c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee67c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49284/0 (socket says 172.21.15.47:49284) 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 -- 172.21.15.47:0/1595664277 learned_addr learned my addr 172.21.15.47:0/1595664277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee66ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee679910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 -- 172.21.15.47:0/1595664277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 msgr2=0x55dbee66ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee66ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 -- 172.21.15.47:0/1595664277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 msgr2=0x55dbee679910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee679910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 -- 172.21.15.47:0/1595664277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbee52c820 con 0x55dbee679e50 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee66ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee679910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee67c240 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8098980 tx=0x7fb4c809b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4766c5c03edd2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4bcff9700 1 -- 172.21.15.47:0/1595664277 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80b5070 con 0x55dbee679e50 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4bcff9700 1 -- 172.21.15.47:0/1595664277 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c80a4c20 con 0x55dbee679e50 2024-04-25T21:02:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.526+0000 7fb4bcff9700 1 -- 172.21.15.47:0/1595664277 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80bc050 con 0x55dbee679e50 2024-04-25T21:02:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 -- 172.21.15.47:0/1595664277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 msgr2=0x55dbee67c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee67c240 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8098980 tx=0x7fb4c809b9e0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 -- 172.21.15.47:0/1595664277 shutdown_connections 2024-04-25T21:02:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee67c240 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee66ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1595664277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee679910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 -- 172.21.15.47:0/1595664277 >> 172.21.15.47:0/1595664277 conn(0x55dbee59b970 msgr2=0x55dbee66aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 -- 172.21.15.47:0/1595664277 shutdown_connections 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 -- 172.21.15.47:0/1595664277 wait complete. 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.527+0000 7fb4df817b80 1 -- start start 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4df817b80 1 Processor -- start 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66e630 0x55dbee6796d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66ef50 0x55dbee6769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 0x55dbee676f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4df817b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dbee67c8e0 con 0x55dbee66e630 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4df817b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dbee57b690 con 0x55dbee679e50 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4df817b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dbee67ca60 con 0x55dbee66ef50 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ce59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66e630 0x55dbee6796d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4df817b80 10 client.0 _ll_register_callbacks cb 0x55dbee55aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ce59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66e630 0x55dbee6796d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49296/0 (socket says 172.21.15.47:49296) 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ce59c700 1 -- 172.21.15.47:0/1297204291 learned_addr learned my addr 172.21.15.47:0/1297204291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:25.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 0x55dbee676f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66ef50 0x55dbee6769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ce59c700 1 -- 172.21.15.47:0/1297204291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66ef50 msgr2=0x55dbee6769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66ef50 0x55dbee6769d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ce59c700 1 -- 172.21.15.47:0/1297204291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 msgr2=0x55dbee676f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 0x55dbee676f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4ce59c700 1 -- 172.21.15.47:0/1297204291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbee50f9b0 con 0x55dbee66e630 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.528+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66ef50 0x55dbee6769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66e630 0x55dbee6796d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00096a0 tx=0x7fb4c0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3682e30cbd63cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/1297204291 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0034060 con 0x55dbee66e630 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/1297204291 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fb4c000b9d0 con 0x55dbee66e630 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 1 -- 172.21.15.47:0/1297204291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c80a4c20 con 0x55dbee66e630 2024-04-25T21:02:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/1297204291 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4c000bc10 con 0x55dbee66e630 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 10 client.15585 fetch_fsmap learned FSMap version 28 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 1 -- 172.21.15.47:0/1297204291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbee6872a0 con 0x55dbee66e630 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/1297204291 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c80a4c20 con 0x55dbee66e630 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/1297204291 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fb4c002e070 con 0x55dbee66e630 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 10 client.15585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 20 client.15585 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55dbee6898e0 0x55dbee68bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 4 client.15585 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.529+0000 7fb4df817b80 1 -- 172.21.15.47:0/1297204291 --> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbee5cf560 con 0x55dbee6898e0 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.530+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55dbee6898e0 0x55dbee68bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.530+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55dbee6898e0 0x55dbee68bd30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7257fd6c142007c server_cookie=70b21903b649c15a in_seq=0 out_seq=0 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.530+0000 7fb4b5ffb700 10 client.15585 ms_handle_connect on v2:172.21.15.92:6836/1984144918 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/1297204291 <== mds.1 v2:172.21.15.92:6836/1984144918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbee5cf560 con 0x55dbee6898e0 2024-04-25T21:02:25.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4b5ffb700 10 client.15585 handle_command_reply: tid=0 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 client.15585 shutdown 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 -- 172.21.15.47:0/1297204291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66e630 msgr2=0x55dbee6796d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66e630 0x55dbee6796d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00096a0 tx=0x7fb4c0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 -- 172.21.15.47:0/1297204291 shutdown_connections 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55dbee6898e0 0x55dbee68bd30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66e630 0x55dbee6796d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 0x55dbee676f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4b5ffb700 0 client.15585 ms_handle_reset on v2:172.21.15.92:6836/1984144918 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 --2- 172.21.15.47:0/1297204291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66ef50 0x55dbee6769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 -- 172.21.15.47:0/1297204291 >> 172.21.15.47:0/1297204291 conn(0x55dbee59b970 msgr2=0x55dbee670fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 -- 172.21.15.47:0/1297204291 shutdown_connections 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 1 -- 172.21.15.47:0/1297204291 wait complete. 2024-04-25T21:02:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.531+0000 7fb4df817b80 20 client.15585 trim_cache size 0 max 16384 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4df817b80 1 Processor -- start 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4df817b80 1 -- start start 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee6857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66ef50 0x55dbee691690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 0x55dbee691bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4df817b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d01b1e70 con 0x55dbee66ef50 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4df817b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d01af700 con 0x55dbee679e50 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4df817b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d01ad570 con 0x55dbee66e630 2024-04-25T21:02:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4cdd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66ef50 0x55dbee691690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4cdd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66ef50 0x55dbee691690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49304/0 (socket says 172.21.15.47:49304) 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4cdd9b700 1 -- 172.21.15.47:0/881111620 learned_addr learned my addr 172.21.15.47:0/881111620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4cdd9b700 1 -- 172.21.15.47:0/881111620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 msgr2=0x55dbee6857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/881111620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee6857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4cdd9b700 1 -- 172.21.15.47:0/881111620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 msgr2=0x55dbee691bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/881111620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 0x55dbee691bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.549+0000 7fb4cdd9b700 1 -- 172.21.15.47:0/881111620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbee68c3b0 con 0x55dbee66ef50 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/881111620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66ef50 0x55dbee691690 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005d90 tx=0x7fb4c400b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c38be7991e320f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4b7fff700 1 -- 172.21.15.47:0/881111620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c401a530 con 0x55dbee66ef50 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4b7fff700 1 -- 172.21.15.47:0/881111620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c401e030 con 0x55dbee66ef50 2024-04-25T21:02:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4b7fff700 1 -- 172.21.15.47:0/881111620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4005460 con 0x55dbee66ef50 2024-04-25T21:02:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 -- 172.21.15.47:0/881111620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66ef50 msgr2=0x55dbee691690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 --2- 172.21.15.47:0/881111620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66ef50 0x55dbee691690 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005d90 tx=0x7fb4c400b3a0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 -- 172.21.15.47:0/881111620 shutdown_connections 2024-04-25T21:02:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 --2- 172.21.15.47:0/881111620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee679e50 0x55dbee691bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 --2- 172.21.15.47:0/881111620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee6857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 --2- 172.21.15.47:0/881111620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee66ef50 0x55dbee691690 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 -- 172.21.15.47:0/881111620 >> 172.21.15.47:0/881111620 conn(0x55dbee59b970 msgr2=0x55dbee669c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 -- 172.21.15.47:0/881111620 shutdown_connections 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.550+0000 7fb4df817b80 1 -- 172.21.15.47:0/881111620 wait complete. 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 1 -- start start 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 1 Processor -- start 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee681e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee680200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee680740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d01adbc0 con 0x55dbee679e50 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d01af030 con 0x55dbee66ef50 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d01ae290 con 0x55dbee66e630 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4df817b80 10 client.0 _ll_register_callbacks cb 0x55dbee55aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee680740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee680740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49312/0 (socket says 172.21.15.47:49312) 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4ced9d700 1 -- 172.21.15.47:0/2642801671 learned_addr learned my addr 172.21.15.47:0/2642801671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee681e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.551+0000 7fb4cdd9b700 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee680200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4ced9d700 1 -- 172.21.15.47:0/2642801671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 msgr2=0x55dbee681e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee681e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4ced9d700 1 -- 172.21.15.47:0/2642801671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 msgr2=0x55dbee680200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee680200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4ced9d700 1 -- 172.21.15.47:0/2642801671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbee680d50 con 0x55dbee679e50 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4ced9d700 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee680740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80cdd60 tx=0x7fb4c8098250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6efe650444df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/2642801671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80b5040 con 0x55dbee679e50 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/2642801671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fb4c809c210 con 0x55dbee679e50 2024-04-25T21:02:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4df817b80 1 -- 172.21.15.47:0/2642801671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c401e030 con 0x55dbee679e50 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/2642801671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4c80bc020 con 0x55dbee679e50 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4df817b80 10 client.15591 fetch_fsmap learned FSMap version 28 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4df817b80 1 -- 172.21.15.47:0/2642801671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbee6f0d50 con 0x55dbee679e50 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.552+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/2642801671 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c401e030 con 0x55dbee679e50 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/2642801671 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fb4c809b400 con 0x55dbee679e50 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4df817b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4df817b80 10 client.15591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4df817b80 20 client.15591 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4df817b80 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55dbee6f0fd0 0x55dbee6f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4df817b80 4 client.15591 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-9ebddfb8-1-mds.h.json"}] 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4df817b80 1 -- 172.21.15.47:0/2642801671 --> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-9ebddfb8-1-mds.h.json"}) -- 0x55dbee5c9600 con 0x55dbee6f0fd0 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55dbee6f0fd0 0x55dbee6f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4ce59c700 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55dbee6f0fd0 0x55dbee6f33c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a79c21b0586185fd server_cookie=283887a80dc2db1e in_seq=0 out_seq=0 2024-04-25T21:02:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.553+0000 7fb4b5ffb700 10 client.15591 ms_handle_connect on v2:172.21.15.92:6836/1984144918 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.554+0000 7fb4b5ffb700 1 -- 172.21.15.47:0/2642801671 <== mds.1 v2:172.21.15.92:6836/1984144918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x55dbee5c9600 con 0x55dbee6f0fd0 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.554+0000 7fb4b5ffb700 10 client.15591 handle_command_reply: tid=0 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.554+0000 7fb4df817b80 1 client.15591 shutdown 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 -- 172.21.15.47:0/2642801671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 msgr2=0x55dbee680740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee680740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80cdd60 tx=0x7fb4c8098250 comp rx=0 tx=0).stop 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 -- 172.21.15.47:0/2642801671 shutdown_connections 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.92:6836/1984144918,v1:172.21.15.92:6837/1984144918] conn(0x55dbee6f0fd0 0x55dbee6f33c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dbee679e50 0x55dbee680740 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dbee66e630 0x55dbee681e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 --2- 172.21.15.47:0/2642801671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbee66ef50 0x55dbee680200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 -- 172.21.15.47:0/2642801671 >> 172.21.15.47:0/2642801671 conn(0x55dbee59b970 msgr2=0x55dbee678200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4b5ffb700 0 client.15591 ms_handle_reset on v2:172.21.15.92:6836/1984144918 2024-04-25T21:02:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 -- 172.21.15.47:0/2642801671 shutdown_connections 2024-04-25T21:02:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 1 -- 172.21.15.47:0/2642801671 wait complete. 2024-04-25T21:02:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4df817b80 20 client.15591 trim_cache size 0 max 16384 2024-04-25T21:02:25.561 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:02:25.561 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/var/run/ceph/ops-9ebddfb8-1-mds.h.json" 2024-04-25T21:02:25.561 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4b80a3db0 msgr2=0x7fb4b80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4b80a3db0 0x7fb4b80a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0006010 tx=0x7fb4c000b540 comp rx=0 tx=0).stop 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 msgr2=0x7fb4d00757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 0x7fb4d00757c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8007ee0 tx=0x7fb4c800d3b0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/833553107 shutdown_connections 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4b80a3db0 0x7fb4b80a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d0108a50 0x7fb4d00757c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0102a90 0x7fb4d0074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 --2- 172.21.15.47:0/833553107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d0103450 0x7fb4d0075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.555+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/833553107 >> 172.21.15.47:0/833553107 conn(0x7fb4d0079ba0 msgr2=0x7fb4d0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.556+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/833553107 shutdown_connections 2024-04-25T21:02:25.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:25.556+0000 7fb4d4f8e700 1 -- 172.21.15.47:0/833553107 wait complete. 2024-04-25T21:02:26.380 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.b.json 2024-04-25T21:02:26.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph tell mds.1:0 ops --flags=locks --path=/var/run/ceph/ops-9ebddfb8-1-mds.b.json 2024-04-25T21:02:26.522 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:26 smithi186 ceph-mon[32529]: pgmap v76: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-25T21:02:26.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:26 smithi092 ceph-mon[32524]: pgmap v76: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-25T21:02:26.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:26 smithi047 ceph-mon[29195]: pgmap v76: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-25T21:02:26.631 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:02:28.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 -- 172.21.15.47:0/205859742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac107990 msgr2=0x7f30ac107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:28.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 --2- 172.21.15.47:0/205859742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac107990 0x7f30ac107d70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009990 tx=0x7f30a8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 -- 172.21.15.47:0/205859742 shutdown_connections 2024-04-25T21:02:28.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 --2- 172.21.15.47:0/205859742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac107990 0x7f30ac107d70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 --2- 172.21.15.47:0/205859742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ac102300 0x7f30ac10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 --2- 172.21.15.47:0/205859742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30ac101940 0x7f30ac101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 -- 172.21.15.47:0/205859742 >> 172.21.15.47:0/205859742 conn(0x7f30ac0788f0 msgr2=0x7f30ac078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 -- 172.21.15.47:0/205859742 shutdown_connections 2024-04-25T21:02:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.496+0000 7f30b3d12700 1 -- 172.21.15.47:0/205859742 wait complete. 2024-04-25T21:02:28.498 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:28 smithi047 ceph-mon[29195]: pgmap v77: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:28.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.497+0000 7f30b3d12700 1 Processor -- start 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.497+0000 7f30b3d12700 1 -- start start 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.497+0000 7f30b3d12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ac101940 0x7f30ac199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.497+0000 7f30b3d12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac102300 0x7f30ac19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.497+0000 7f30b3d12700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30ac107990 0x7f30ac1a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.497+0000 7f30b3d12700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac111b00 con 0x7f30ac102300 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.497+0000 7f30b3d12700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac111980 con 0x7f30ac101940 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.497+0000 7f30b3d12700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac111c80 con 0x7f30ac107990 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac102300 0x7f30ac19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ac101940 0x7f30ac199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac102300 0x7f30ac19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49328/0 (socket says 172.21.15.47:49328) 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 -- 172.21.15.47:0/1890748793 learned_addr learned my addr 172.21.15.47:0/1890748793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b22af700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30ac107990 0x7f30ac1a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 -- 172.21.15.47:0/1890748793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30ac107990 msgr2=0x7f30ac1a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:28.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30ac107990 0x7f30ac1a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 -- 172.21.15.47:0/1890748793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ac101940 msgr2=0x7f30ac199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ac101940 0x7f30ac199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 -- 172.21.15.47:0/1890748793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac1a5500 con 0x7f30ac102300 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b12ad700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac102300 0x7f30ac19a480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c970 tx=0x7f309c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30a2ffd700 1 -- 172.21.15.47:0/1890748793 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c007aa0 con 0x7f30ac102300 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30b3d12700 1 -- 172.21.15.47:0/1890748793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac1a5790 con 0x7f30ac102300 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.498+0000 7f30a2ffd700 1 -- 172.21.15.47:0/1890748793 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f309c004ce0 con 0x7f30ac102300 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.499+0000 7f30a2ffd700 1 -- 172.21.15.47:0/1890748793 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c005510 con 0x7f30ac102300 2024-04-25T21:02:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.499+0000 7f30b3d12700 1 -- 172.21.15.47:0/1890748793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac1a5c50 con 0x7f30ac102300 2024-04-25T21:02:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.501+0000 7f30a2ffd700 1 -- 172.21.15.47:0/1890748793 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f309c007c40 con 0x7f30ac102300 2024-04-25T21:02:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.501+0000 7f30a2ffd700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f30980a3db0 0x7f30980a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.502+0000 7f30b1aae700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f30980a3db0 0x7f30980a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.502+0000 7f30a2ffd700 1 -- 172.21.15.47:0/1890748793 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f309c0d0560 con 0x7f30ac102300 2024-04-25T21:02:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.502+0000 7f30b1aae700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f30980a3db0 0x7f30980a6270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b5c0 tx=0x7f30a8005960 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.503+0000 7f30be59bb80 1 Processor -- start 2024-04-25T21:02:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.503+0000 7f30be59bb80 1 -- start start 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.503+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.503+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 0x55875d069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.503+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 0x55875d06c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30be59bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55875cfb1e10 con 0x55875d05e9d0 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30be59bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55875cf58a40 con 0x55875d05f2f0 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30be59bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55875cfaddd0 con 0x55875d06a1c0 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49338/0 (socket says 172.21.15.47:49338) 2024-04-25T21:02:28.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 -- 172.21.15.47:0/2414408680 learned_addr learned my addr 172.21.15.47:0/2414408680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b22af700 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 0x55875d06c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b12ad700 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 0x55875d069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 -- 172.21.15.47:0/2414408680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 msgr2=0x55875d06c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 0x55875d06c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 -- 172.21.15.47:0/2414408680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 msgr2=0x55875d069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 0x55875d069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 -- 172.21.15.47:0/2414408680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55875ceff9b0 con 0x55875d05e9d0 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f30b1aae700 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f30a8015000 tx=0x7f30a801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215b0b373b0f0869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f3097fff700 1 -- 172.21.15.47:0/2414408680 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8034070 con 0x55875d05e9d0 2024-04-25T21:02:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f3097fff700 1 -- 172.21.15.47:0/2414408680 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30a8024930 con 0x55875d05e9d0 2024-04-25T21:02:28.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.504+0000 7f3097fff700 1 -- 172.21.15.47:0/2414408680 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a802fab0 con 0x55875d05e9d0 2024-04-25T21:02:28.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 -- 172.21.15.47:0/2414408680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 msgr2=0x55875d05edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:28.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f30a8015000 tx=0x7f30a801f000 comp rx=0 tx=0).stop 2024-04-25T21:02:28.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 -- 172.21.15.47:0/2414408680 shutdown_connections 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05edb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 0x55875d06c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 --2- 172.21.15.47:0/2414408680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 0x55875d069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 -- 172.21.15.47:0/2414408680 >> 172.21.15.47:0/2414408680 conn(0x55875cf8c770 msgr2=0x55875d05aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 -- 172.21.15.47:0/2414408680 shutdown_connections 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.505+0000 7f30be59bb80 1 -- 172.21.15.47:0/2414408680 wait complete. 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 1 -- start start 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 1 Processor -- start 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d0698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 0x55875d066be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 0x55875d067120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55875d06cb50 con 0x55875d05e9d0 2024-04-25T21:02:28.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55875cf50cb0 con 0x55875d05f2f0 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55875d06ccd0 con 0x55875d06a1c0 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30be59bb80 10 client.0 _ll_register_callbacks cb 0x55875cf4aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d0698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d0698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49346/0 (socket says 172.21.15.47:49346) 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30b1aae700 1 -- 172.21.15.47:0/12523392 learned_addr learned my addr 172.21.15.47:0/12523392 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30b12ad700 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 0x55875d066be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.506+0000 7f30b22af700 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 0x55875d067120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30b1aae700 1 -- 172.21.15.47:0/12523392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 msgr2=0x55875d067120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30b1aae700 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 0x55875d067120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30b1aae700 1 -- 172.21.15.47:0/12523392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 msgr2=0x55875d066be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30b1aae700 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 0x55875d066be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30b1aae700 1 -- 172.21.15.47:0/12523392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55875cf69350 con 0x55875d05e9d0 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30b1aae700 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d0698e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f30a8032040 tx=0x7f30a8024d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814c2777d837df36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:28.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30957fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30957fa700 1 -- 172.21.15.47:0/12523392 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8034040 con 0x55875d05e9d0 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30957fa700 1 -- 172.21.15.47:0/12523392 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f30a802f730 con 0x55875d05e9d0 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30be59bb80 1 -- 172.21.15.47:0/12523392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a8024930 con 0x55875d05e9d0 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30957fa700 1 -- 172.21.15.47:0/12523392 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30a802f930 con 0x55875d05e9d0 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30be59bb80 10 client.15603 fetch_fsmap learned FSMap version 28 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30be59bb80 1 -- 172.21.15.47:0/12523392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55875d0773a0 con 0x55875d05e9d0 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.507+0000 7f30957fa700 1 -- 172.21.15.47:0/12523392 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30a8024930 con 0x55875d05e9d0 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30957fa700 1 -- 172.21.15.47:0/12523392 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f30a803aa00 con 0x55875d05e9d0 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30be59bb80 10 client.15603 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30be59bb80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30be59bb80 20 client.15603 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30be59bb80 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55875d079870 0x55875d07bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:28.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30be59bb80 4 client.15603 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:28.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30be59bb80 1 -- 172.21.15.47:0/12523392 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55875cfbf600 con 0x55875d079870 2024-04-25T21:02:28.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30b12ad700 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55875d079870 0x55875d07bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:28.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30b12ad700 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55875d079870 0x55875d07bc60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c41cd683721e649 server_cookie=af0b941786239609 in_seq=0 out_seq=0 2024-04-25T21:02:28.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:28.508+0000 7f30957fa700 10 client.15603 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:28.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:28 smithi092 ceph-mon[32524]: pgmap v77: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:28.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:28 smithi186 ceph-mon[32529]: pgmap v77: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:30.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:30 smithi092 ceph-mon[32524]: pgmap v78: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:30.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:30 smithi047 ceph-mon[29195]: pgmap v78: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:30 smithi186 ceph-mon[32529]: pgmap v78: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:32.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:32 smithi092 ceph-mon[32524]: pgmap v79: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T21:02:32.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:32 smithi047 ceph-mon[29195]: pgmap v79: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T21:02:32.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:32 smithi186 ceph-mon[32529]: pgmap v79: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T21:02:34.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:34 smithi092 ceph-mon[32524]: pgmap v80: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:34.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:34 smithi047 ceph-mon[29195]: pgmap v80: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:34.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:34 smithi186 ceph-mon[32529]: pgmap v80: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-25T21:02:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cceff5640 1 -- 172.21.15.47:0/2457424110 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4b2c1e200 con 0x55c4b2d33090 2024-04-25T21:02:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cceff5640 10 client.15549 handle_command_reply: tid=0 2024-04-25T21:02:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 client.15549 shutdown 2024-04-25T21:02:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 -- 172.21.15.47:0/2457424110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 msgr2=0x55c4b2cc1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cc1fb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005b60 tx=0x7f5ce000cdd0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 -- 172.21.15.47:0/2457424110 shutdown_connections 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c4b2d33090 0x55c4b2d354b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4b2cbc800 0x55c4b2cc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4b2cb1f40 0x55c4b2cc1fb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 --2- 172.21.15.47:0/2457424110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4b2cb1620 0x55c4b2cc4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 -- 172.21.15.47:0/2457424110 >> 172.21.15.47:0/2457424110 conn(0x55c4b2cadb30 msgr2=0x55c4b2caccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cceff5640 0 client.15549 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f5cf41a9740 1 -- 172.21.15.47:0/2457424110 shutdown_connections 2024-04-25T21:02:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f5cf41a9740 1 -- 172.21.15.47:0/2457424110 wait complete. 2024-04-25T21:02:35.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f30957fa700 1 -- 172.21.15.47:0/12523392 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55875cfbf600 con 0x55875d079870 2024-04-25T21:02:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f30957fa700 10 client.15603 handle_command_reply: tid=0 2024-04-25T21:02:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f5cf41a9740 20 client.15549 trim_cache size 0 max 16384 2024-04-25T21:02:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.539+0000 7f30be59bb80 1 client.15603 shutdown 2024-04-25T21:02:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 -- 172.21.15.47:0/12523392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 msgr2=0x55875d0698e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d0698e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f30a8032040 tx=0x7f30a8024d10 comp rx=0 tx=0).stop 2024-04-25T21:02:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 -- 172.21.15.47:0/12523392 shutdown_connections 2024-04-25T21:02:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55875d079870 0x55875d07bc60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d0698e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d06a1c0 0x55875d067120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 --2- 172.21.15.47:0/12523392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05f2f0 0x55875d066be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 -- 172.21.15.47:0/12523392 >> 172.21.15.47:0/12523392 conn(0x55875cf8c770 msgr2=0x55875d05aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30957fa700 0 client.15603 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:02:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:02:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:02:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:02:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 -- 172.21.15.47:0/12523392 shutdown_connections 2024-04-25T21:02:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 1 -- 172.21.15.47:0/12523392 wait complete. 2024-04-25T21:02:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f30be59bb80 20 client.15603 trim_cache size 0 max 16384 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f5cf1491640 1 -- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5cd40a3e40 msgr2=0x7f5cd40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f5cf1491640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5cd40a3e40 0x7f5cd40a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4004af0 tx=0x7f5ce40095c0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f5cf1491640 1 -- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1010e0 msgr2=0x7f5cec10e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.540+0000 7f5cf1491640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1010e0 0x7f5cec10e710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00d950 tx=0x7f5cdc00de10 comp rx=0 tx=0).stop 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.541+0000 7f5cf1491640 1 -- 172.21.15.47:0/3844434620 shutdown_connections 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.541+0000 7f5cf1491640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5cd40a3e40 0x7f5cd40a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.541+0000 7f5cf1491640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cec10f190 0x7f5cec1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.541+0000 7f5cf1491640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec1061b0 0x7f5cec10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.541+0000 7f5cf1491640 1 --2- 172.21.15.47:0/3844434620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cec1010e0 0x7f5cec10e710 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.541+0000 7f5cf1491640 1 -- 172.21.15.47:0/3844434620 >> 172.21.15.47:0/3844434620 conn(0x7f5cec0771d0 msgr2=0x7f5cec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.541+0000 7f5cf1491640 1 -- 172.21.15.47:0/3844434620 shutdown_connections 2024-04-25T21:02:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:35.541+0000 7f5cf1491640 1 -- 172.21.15.47:0/3844434620 wait complete. 2024-04-25T21:02:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.567+0000 7f30be59bb80 1 Processor -- start 2024-04-25T21:02:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.567+0000 7f30be59bb80 1 -- start start 2024-04-25T21:02:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d069420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d06a1c0 0x55875d07e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30be59bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1afa50 con 0x55875d05e9d0 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30be59bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1ad2e0 con 0x55875d06a1c0 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30be59bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1ab150 con 0x55875d05f2f0 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34228/0 (socket says 172.21.15.47:34228) 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 -- 172.21.15.47:0/3965729711 learned_addr learned my addr 172.21.15.47:0/3965729711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b22af700 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d06a1c0 0x55875d07e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b12ad700 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 -- 172.21.15.47:0/3965729711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 msgr2=0x55875d069420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d069420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 -- 172.21.15.47:0/3965729711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d06a1c0 msgr2=0x55875d07e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d06a1c0 0x55875d07e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 -- 172.21.15.47:0/3965729711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55875d06cb20 con 0x55875d05e9d0 2024-04-25T21:02:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b22af700 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d06a1c0 0x55875d07e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.568+0000 7f30b1aae700 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05b040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009750 tx=0x7f30a80173f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c005e6a4875e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.569+0000 7f30977fe700 1 -- 172.21.15.47:0/3965729711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8046070 con 0x55875d05e9d0 2024-04-25T21:02:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.569+0000 7f30977fe700 1 -- 172.21.15.47:0/3965729711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30a8024d80 con 0x55875d05e9d0 2024-04-25T21:02:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.569+0000 7f30977fe700 1 -- 172.21.15.47:0/3965729711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8034070 con 0x55875d05e9d0 2024-04-25T21:02:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 -- 172.21.15.47:0/3965729711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 msgr2=0x55875d05b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05b040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009750 tx=0x7f30a80173f0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 -- 172.21.15.47:0/3965729711 shutdown_connections 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d05e9d0 0x55875d05b040 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d06a1c0 0x55875d07e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 --2- 172.21.15.47:0/3965729711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d069420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 -- 172.21.15.47:0/3965729711 >> 172.21.15.47:0/3965729711 conn(0x55875cf8c770 msgr2=0x55875d066490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 -- 172.21.15.47:0/3965729711 shutdown_connections 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 -- 172.21.15.47:0/3965729711 wait complete. 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 -- start start 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.570+0000 7f30be59bb80 1 Processor -- start 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05e9d0 0x55875d07a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d07aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30be59bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d06a1c0 0x55875d07afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30be59bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1ab7a0 con 0x55875d06a1c0 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30be59bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1acc10 con 0x55875d05e9d0 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30be59bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1abe70 con 0x55875d05f2f0 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30be59bb80 10 client.0 _ll_register_callbacks cb 0x55875cf4aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b22af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d06a1c0 0x55875d07afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05e9d0 0x55875d07a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b12ad700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d07aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b1aae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05e9d0 0x55875d07a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:52468/0 (socket says 172.21.15.47:52468) 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b22af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d06a1c0 0x55875d07afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34234/0 (socket says 172.21.15.47:34234) 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b1aae700 1 -- 172.21.15.47:0/1614381668 learned_addr learned my addr 172.21.15.47:0/1614381668 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b1aae700 1 -- 172.21.15.47:0/1614381668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 msgr2=0x55875d07aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b1aae700 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d07aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b1aae700 1 -- 172.21.15.47:0/1614381668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d06a1c0 msgr2=0x55875d07afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b1aae700 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d06a1c0 0x55875d07afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b1aae700 1 -- 172.21.15.47:0/1614381668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55875d0decf0 con 0x55875d05e9d0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b12ad700 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d07aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.571+0000 7f30b22af700 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d06a1c0 0x55875d07afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30b1aae700 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05e9d0 0x55875d07a540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c010 tx=0x7f30a80175c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=235d6b2e86da6e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30957fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30957fa700 1 -- 172.21.15.47:0/1614381668 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a804e020 con 0x55875d05e9d0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30957fa700 1 -- 172.21.15.47:0/1614381668 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f30a802fad0 con 0x55875d05e9d0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 1 -- 172.21.15.47:0/1614381668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a8024d80 con 0x55875d05e9d0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30957fa700 1 -- 172.21.15.47:0/1614381668 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30a8059050 con 0x55875d05e9d0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 10 client.25238 fetch_fsmap learned FSMap version 28 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 1 -- 172.21.15.47:0/1614381668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55875d0dffb0 con 0x55875d05e9d0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30957fa700 1 -- 172.21.15.47:0/1614381668 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30a8024d80 con 0x55875d05e9d0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30957fa700 1 -- 172.21.15.47:0/1614381668 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f30a803a430 con 0x55875d05e9d0 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 10 client.25238 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 10 client.25238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 20 client.25238 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:35.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55875d0e04e0 0x55875d0e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 4 client.25238 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-9ebddfb8-1-mds.b.json"}] 2024-04-25T21:02:35.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.572+0000 7f30be59bb80 1 -- 172.21.15.47:0/1614381668 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-9ebddfb8-1-mds.b.json"}) -- 0x55875cfb96a0 con 0x55875d0e04e0 2024-04-25T21:02:35.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.573+0000 7f30b12ad700 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55875d0e04e0 0x55875d0e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.573+0000 7f30957fa700 10 client.25238 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:35.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.573+0000 7f30b12ad700 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55875d0e04e0 0x55875d0e2900 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a773f1493bfed30e server_cookie=1c3f63521a264bfa in_seq=0 out_seq=0 2024-04-25T21:02:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30957fa700 1 -- 172.21.15.47:0/1614381668 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x55875cfb96a0 con 0x55875d0e04e0 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30957fa700 10 client.25238 handle_command_reply: tid=0 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 client.25238 shutdown 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 -- 172.21.15.47:0/1614381668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05e9d0 msgr2=0x55875d07a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05e9d0 0x55875d07a540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c010 tx=0x7f30a80175c0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 -- 172.21.15.47:0/1614381668 shutdown_connections 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55875d0e04e0 0x55875d0e2900 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55875d06a1c0 0x55875d07afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d05e9d0 0x55875d07a540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 --2- 172.21.15.47:0/1614381668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55875d05f2f0 0x55875d07aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30957fa700 0 client.25238 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 -- 172.21.15.47:0/1614381668 >> 172.21.15.47:0/1614381668 conn(0x55875cf8c770 msgr2=0x55875d074f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 -- 172.21.15.47:0/1614381668 shutdown_connections 2024-04-25T21:02:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 1 -- 172.21.15.47:0/1614381668 wait complete. 2024-04-25T21:02:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.585+0000 7f30be59bb80 20 client.25238 trim_cache size 0 max 16384 2024-04-25T21:02:35.587 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:02:35.587 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/var/run/ceph/ops-9ebddfb8-1-mds.b.json" 2024-04-25T21:02:35.587 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 -- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f30980a3db0 msgr2=0x7f30980a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f30980a3db0 0x7f30980a6270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b5c0 tx=0x7f30a8005960 comp rx=0 tx=0).stop 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 -- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac102300 msgr2=0x7f30ac19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac102300 0x7f30ac19a480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c970 tx=0x7f309c00cc80 comp rx=0 tx=0).stop 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 -- 172.21.15.47:0/1890748793 shutdown_connections 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f30980a3db0 0x7f30980a6270 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30ac107990 0x7f30ac1a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ac101940 0x7f30ac199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 --2- 172.21.15.47:0/1890748793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ac102300 0x7f30ac19a480 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 -- 172.21.15.47:0/1890748793 >> 172.21.15.47:0/1890748793 conn(0x7f30ac0788f0 msgr2=0x7f30ac100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 -- 172.21.15.47:0/1890748793 shutdown_connections 2024-04-25T21:02:35.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:35.586+0000 7f30b3d12700 1 -- 172.21.15.47:0/1890748793 wait complete. 2024-04-25T21:02:36.327 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-04-25T21:02:36.345 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.C4bi5cmXV8 2024-04-25T21:02:36.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.e.json /tmp/tmp.C4bi5cmXV8 2024-04-25T21:02:36.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.C4bi5cmXV8 2024-04-25T21:02:36.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:36 smithi092 ceph-mon[32524]: pgmap v81: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T21:02:36.542 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.C4bi5cmXV8 is 24KB 2024-04-25T21:02:36.592 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.C4bi5cmXV8 2024-04-25T21:02:36.607 DEBUG:tasks.quiescer.fs.[cephfs]:read 24384B of ops from '/var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.e.json' on mds.2 (e) 2024-04-25T21:02:36.612 INFO:tasks.quiescer.fs.[cephfs]:Pulled 14 ops from rank 2 (e) into home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/ops-9ebddfb8-1-mds.e.json 2024-04-25T21:02:36.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -- /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.e.json 2024-04-25T21:02:36.612 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:36 smithi047 ceph-mon[29195]: pgmap v81: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T21:02:36.672 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-04-25T21:02:36.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:36 smithi186 ceph-mon[32529]: pgmap v81: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T21:02:36.727 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.49fpKqDSN2 2024-04-25T21:02:36.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.h.json /tmp/tmp.49fpKqDSN2 2024-04-25T21:02:36.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.49fpKqDSN2 2024-04-25T21:02:36.915 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.49fpKqDSN2 is 24KB 2024-04-25T21:02:36.965 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.49fpKqDSN2 2024-04-25T21:02:36.979 DEBUG:tasks.quiescer.fs.[cephfs]:read 24396B of ops from '/var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.h.json' on mds.1 (h) 2024-04-25T21:02:36.990 INFO:tasks.quiescer.fs.[cephfs]:Pulled 14 ops from rank 1 (h) into home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/ops-9ebddfb8-1-mds.h.json 2024-04-25T21:02:36.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -- /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.h.json 2024-04-25T21:02:37.043 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-04-25T21:02:37.098 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.dvj9dCRKeL 2024-04-25T21:02:37.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.b.json /tmp/tmp.dvj9dCRKeL 2024-04-25T21:02:37.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.dvj9dCRKeL 2024-04-25T21:02:37.282 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.dvj9dCRKeL is 365KB 2024-04-25T21:02:37.376 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.dvj9dCRKeL 2024-04-25T21:02:37.392 DEBUG:tasks.quiescer.fs.[cephfs]:read 373477B of ops from '/var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.b.json' on mds.0 (b) 2024-04-25T21:02:37.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:782 create owner_uid=1000, owner_gid=1316 #0x100000003a4/008 2024-04-25T21:02:35.555740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:783 create owner_uid=1000, owner_gid=1316 #0x100000003a4/008.out 2024-04-25T21:02:35.557740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:784 create owner_uid=1000, owner_gid=1316 #0x100000003a4/009 2024-04-25T21:02:35.559740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:785 create owner_uid=1000, owner_gid=1316 #0x100000003a4/009.out 2024-04-25T21:02:35.562740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:786 create owner_uid=1000, owner_gid=1316 #0x100000003a4/010 2024-04-25T21:02:35.564740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:787 create owner_uid=1000, owner_gid=1316 #0x100000003a4/010.out 2024-04-25T21:02:35.566740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:788 create owner_uid=1000, owner_gid=1316 #0x100000003a4/011 2024-04-25T21:02:35.567740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:772 create owner_uid=1000, owner_gid=1316 #0x100000003a4/003 2024-04-25T21:02:35.533740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:773 create owner_uid=1000, owner_gid=1316 #0x100000003a4/003.out 2024-04-25T21:02:35.536740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:774 create owner_uid=1000, owner_gid=1316 #0x100000003a4/004 2024-04-25T21:02:35.537740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:775 create owner_uid=1000, owner_gid=1316 #0x100000003a4/004.out 2024-04-25T21:02:35.539740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:776 create owner_uid=1000, owner_gid=1316 #0x100000003a4/005 2024-04-25T21:02:35.540740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:777 create owner_uid=1000, owner_gid=1316 #0x100000003a4/005.out 2024-04-25T21:02:35.543740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:778 create owner_uid=1000, owner_gid=1316 #0x100000003a4/006 2024-04-25T21:02:35.544740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:779 create owner_uid=1000, owner_gid=1316 #0x100000003a4/006.out 2024-04-25T21:02:35.545740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:780 create owner_uid=1000, owner_gid=1316 #0x100000003a4/007 2024-04-25T21:02:35.547740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 9ebddfb8-1: 'client_request(client.24832:781 create owner_uid=1000, owner_gid=1316 #0x100000003a4/007.out 2024-04-25T21:02:35.549740+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:02:37.425 INFO:tasks.quiescer.fs.[cephfs]:Pulled 17 ops from rank 0 (b) into home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/ops-9ebddfb8-1-mds.b.json 2024-04-25T21:02:37.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -- /var/run/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ops-9ebddfb8-1-mds.b.json 2024-04-25T21:02:37.455 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:02:37.717 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:02:38.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:38 smithi092 ceph-mon[32524]: pgmap v82: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:38.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:38 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:38.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:38 smithi047 ceph-mon[29195]: pgmap v82: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:38.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:38 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:38.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:38 smithi186 ceph-mon[32529]: pgmap v82: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:38.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:38 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 -- 172.21.15.47:0/1468200660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d0075590 msgr2=0x7fc4d010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/1468200660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d0075590 0x7fc4d010ef70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009990 tx=0x7fc4c4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 -- 172.21.15.47:0/1468200660 shutdown_connections 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/1468200660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d010f4b0 0x7fc4d01118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/1468200660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0074c70 0x7fc4d0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/1468200660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d0075590 0x7fc4d010ef70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 -- 172.21.15.47:0/1468200660 >> 172.21.15.47:0/1468200660 conn(0x7fc4d0079c00 msgr2=0x7fc4d007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 -- 172.21.15.47:0/1468200660 shutdown_connections 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.465+0000 7fc4d4b25700 1 -- 172.21.15.47:0/1468200660 wait complete. 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4d4b25700 1 Processor -- start 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4d4b25700 1 -- start start 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4d4b25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0074c70 0x7fc4d019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:39.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4d4b25700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d0075590 0x7fc4d019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4d4b25700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d010f4b0 0x7fc4d019fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4d4b25700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0114dd0 con 0x7fc4d010f4b0 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4d4b25700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0114c50 con 0x7fc4d0074c70 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4d4b25700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0114f50 con 0x7fc4d0075590 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d010f4b0 0x7fc4d019fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d010f4b0 0x7fc4d019fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34254/0 (socket says 172.21.15.47:34254) 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4ced9d700 1 -- 172.21.15.47:0/504765591 learned_addr learned my addr 172.21.15.47:0/504765591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.466+0000 7fc4cdd9b700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d0075590 0x7fc4d019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4ce59c700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0074c70 0x7fc4d019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4ced9d700 1 -- 172.21.15.47:0/504765591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d0075590 msgr2=0x7fc4d019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4ced9d700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d0075590 0x7fc4d019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4ced9d700 1 -- 172.21.15.47:0/504765591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0074c70 msgr2=0x7fc4d019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4ced9d700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0074c70 0x7fc4d019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4ced9d700 1 -- 172.21.15.47:0/504765591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d01a0250 con 0x7fc4d010f4b0 2024-04-25T21:02:39.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4ce59c700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0074c70 0x7fc4d019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4ced9d700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d010f4b0 0x7fc4d019fb10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8007d70 tx=0x7fc4c800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4bf7fe700 1 -- 172.21.15.47:0/504765591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8016020 con 0x7fc4d010f4b0 2024-04-25T21:02:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4bf7fe700 1 -- 172.21.15.47:0/504765591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4c800ad40 con 0x7fc4d010f4b0 2024-04-25T21:02:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d01a0540 con 0x7fc4d010f4b0 2024-04-25T21:02:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4bf7fe700 1 -- 172.21.15.47:0/504765591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c80115e0 con 0x7fc4d010f4b0 2024-04-25T21:02:39.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.467+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d01a7dc0 con 0x7fc4d010f4b0 2024-04-25T21:02:39.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.468+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d0069730 con 0x7fc4d010f4b0 2024-04-25T21:02:39.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.469+0000 7fc4bf7fe700 1 -- 172.21.15.47:0/504765591 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc4c8019020 con 0x7fc4d010f4b0 2024-04-25T21:02:39.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.470+0000 7fc4bf7fe700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc4b80a3e30 0x7fc4b80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:39.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.470+0000 7fc4ce59c700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc4b80a3e30 0x7fc4b80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:39.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.470+0000 7fc4bf7fe700 1 -- 172.21.15.47:0/504765591 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fc4c80d0b90 con 0x7fc4d010f4b0 2024-04-25T21:02:39.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.471+0000 7fc4ce59c700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc4b80a3e30 0x7fc4b80a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005fd0 tx=0x7fc4c0005e30 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.474+0000 7fc4bf7fe700 1 -- 172.21.15.47:0/504765591 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4c8020020 con 0x7fc4d010f4b0 2024-04-25T21:02:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.727+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4d002df80 con 0x7fc4d010f4b0 2024-04-25T21:02:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.728+0000 7fc4bf7fe700 1 -- 172.21.15.47:0/504765591 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11857 (secure 0 0 0) 0x7fc4c809d2f0 con 0x7fc4d010f4b0 2024-04-25T21:02:39.729 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:02:39.729 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T21:02:36.595671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24488,24467]},"id":1}]} 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc4b80a3e30 msgr2=0x7fc4b80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc4b80a3e30 0x7fc4b80a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005fd0 tx=0x7fc4c0005e30 comp rx=0 tx=0).stop 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d010f4b0 msgr2=0x7fc4d019fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d010f4b0 0x7fc4d019fb10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8007d70 tx=0x7fc4c800a340 comp rx=0 tx=0).stop 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 shutdown_connections 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc4b80a3e30 0x7fc4b80a62f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d010f4b0 0x7fc4d019fb10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0074c70 0x7fc4d019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 --2- 172.21.15.47:0/504765591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d0075590 0x7fc4d019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 >> 172.21.15.47:0/504765591 conn(0x7fc4d0079c00 msgr2=0x7fc4d00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 shutdown_connections 2024-04-25T21:02:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:39.730+0000 7fc4d4b25700 1 -- 172.21.15.47:0/504765591 wait complete. 2024-04-25T21:02:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-04-25T21:02:40.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:40 smithi047 ceph-mon[29195]: pgmap v83: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:40.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:40 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/504765591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:02:40.470 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 9ebddfb8 --await-for 42.0' 2024-04-25T21:02:40.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24488 quiesce db --set-id 9ebddfb8 --await-for 42.0 2024-04-25T21:02:40.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:40 smithi092 ceph-mon[32524]: pgmap v83: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:40.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:40 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/504765591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:02:40.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:40 smithi186 ceph-mon[32529]: pgmap v83: 97 pgs: 97 active+clean; 19 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T21:02:40.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:40 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/504765591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:02:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:02:42.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:42 smithi047 ceph-mon[29195]: pgmap v84: 97 pgs: 97 active+clean; 21 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 202 KiB/s wr, 42 op/s 2024-04-25T21:02:42.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 -- 172.21.15.47:0/714230458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97501022f0 msgr2=0x7f975010e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 --2- 172.21.15.47:0/714230458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97501022f0 0x7f975010e7e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f974800b0a0 tx=0x7f974800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 -- 172.21.15.47:0/714230458 shutdown_connections 2024-04-25T21:02:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 --2- 172.21.15.47:0/714230458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97501022f0 0x7f975010e7e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 --2- 172.21.15.47:0/714230458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9750107980 0x7f9750107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 --2- 172.21.15.47:0/714230458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750101930 0x7f9750101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 -- 172.21.15.47:0/714230458 >> 172.21.15.47:0/714230458 conn(0x7f97500788b0 msgr2=0x7f9750078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 -- 172.21.15.47:0/714230458 shutdown_connections 2024-04-25T21:02:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.448+0000 7f9758787700 1 -- 172.21.15.47:0/714230458 wait complete. 2024-04-25T21:02:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9758787700 1 Processor -- start 2024-04-25T21:02:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9758787700 1 -- start start 2024-04-25T21:02:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9758787700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750101930 0x7f97501034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9758787700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97501022f0 0x7f9750103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9758787700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9750107980 0x7f9750103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9758787700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9750111b40 con 0x7f97501022f0 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9758787700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97501119c0 con 0x7f9750101930 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9758787700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9750111cc0 con 0x7f9750107980 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9755d22700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97501022f0 0x7f9750103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9756523700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750101930 0x7f97501034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9755d22700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97501022f0 0x7f9750103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34284/0 (socket says 172.21.15.47:34284) 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.449+0000 7f9755d22700 1 -- 172.21.15.47:0/547431587 learned_addr learned my addr 172.21.15.47:0/547431587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9756d24700 1 --2- 172.21.15.47:0/547431587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9750107980 0x7f9750103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9755d22700 1 -- 172.21.15.47:0/547431587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9750107980 msgr2=0x7f9750103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9755d22700 1 --2- 172.21.15.47:0/547431587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9750107980 0x7f9750103f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9755d22700 1 -- 172.21.15.47:0/547431587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750101930 msgr2=0x7f97501034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9755d22700 1 --2- 172.21.15.47:0/547431587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750101930 0x7f97501034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9755d22700 1 -- 172.21.15.47:0/547431587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97501a7620 con 0x7f97501022f0 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9756523700 1 --2- 172.21.15.47:0/547431587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750101930 0x7f97501034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9755d22700 1 --2- 172.21.15.47:0/547431587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97501022f0 0x7f9750103a20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f974000e9a0 tx=0x7f974000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f97477fe700 1 -- 172.21.15.47:0/547431587 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974000cbd0 con 0x7f97501022f0 2024-04-25T21:02:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f97477fe700 1 -- 172.21.15.47:0/547431587 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974000cd70 con 0x7f97501022f0 2024-04-25T21:02:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9758787700 1 -- 172.21.15.47:0/547431587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97501a7910 con 0x7f97501022f0 2024-04-25T21:02:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f97477fe700 1 -- 172.21.15.47:0/547431587 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740018680 con 0x7f97501022f0 2024-04-25T21:02:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.450+0000 7f9758787700 1 -- 172.21.15.47:0/547431587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97501a7ea0 con 0x7f97501022f0 2024-04-25T21:02:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.453+0000 7f97477fe700 1 -- 172.21.15.47:0/547431587 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9740010430 con 0x7f97501022f0 2024-04-25T21:02:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.453+0000 7f97477fe700 1 --2- 172.21.15.47:0/547431587 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f973c0a3e30 0x7f973c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.453+0000 7f9756523700 1 --2- 172.21.15.47:0/547431587 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f973c0a3e30 0x7f973c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.453+0000 7f97477fe700 1 -- 172.21.15.47:0/547431587 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f97400d09a0 con 0x7f97501022f0 2024-04-25T21:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9756523700 1 --2- 172.21.15.47:0/547431587 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f973c0a3e30 0x7f973c0a62f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f974c009fd0 tx=0x7f974c009260 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9763010b80 1 Processor -- start 2024-04-25T21:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9763010b80 1 -- start start 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9763010b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528682240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9763010b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x56452868d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9763010b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x56452868fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9763010b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5645285ccd90 con 0x564528681e60 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9763010b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564528573a30 con 0x56452868d660 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.454+0000 7f9763010b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5645285c8d50 con 0x564528682780 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528682240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528682240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34300/0 (socket says 172.21.15.47:34300) 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 -- 172.21.15.47:0/3269474350 learned_addr learned my addr 172.21.15.47:0/3269474350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9755d22700 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x56452868d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756d24700 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x56452868fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 -- 172.21.15.47:0/3269474350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 msgr2=0x56452868d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x56452868d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 -- 172.21.15.47:0/3269474350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 msgr2=0x56452868fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x56452868fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 -- 172.21.15.47:0/3269474350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564528537840 con 0x564528681e60 2024-04-25T21:02:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9756523700 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528682240 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f974c0043c0 tx=0x7f974c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45406474f8063674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9744ff9700 1 -- 172.21.15.47:0/3269474350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c003d10 con 0x564528681e60 2024-04-25T21:02:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9744ff9700 1 -- 172.21.15.47:0/3269474350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974c0268b0 con 0x564528681e60 2024-04-25T21:02:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.455+0000 7f9744ff9700 1 -- 172.21.15.47:0/3269474350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c01e810 con 0x564528681e60 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 -- 172.21.15.47:0/3269474350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 msgr2=0x564528682240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528682240 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f974c0043c0 tx=0x7f974c014000 comp rx=0 tx=0).stop 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 -- 172.21.15.47:0/3269474350 shutdown_connections 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528682240 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x56452868fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 --2- 172.21.15.47:0/3269474350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x56452868d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 -- 172.21.15.47:0/3269474350 >> 172.21.15.47:0/3269474350 conn(0x5645285a6990 msgr2=0x564528675ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 -- 172.21.15.47:0/3269474350 shutdown_connections 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.456+0000 7f9763010b80 1 -- 172.21.15.47:0/3269474350 wait complete. 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 1 -- start start 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 1 Processor -- start 2024-04-25T21:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528681e60 0x56452868cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x56452868a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56452868d660 0x56452868a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564528690180 con 0x56452868d660 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5645285866b0 con 0x564528681e60 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564528690300 con 0x564528682780 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9763010b80 10 client.0 _ll_register_callbacks cb 0x564528565a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756523700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528681e60 0x56452868cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756d24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56452868d660 0x56452868a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756523700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528681e60 0x56452868cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:52494/0 (socket says 172.21.15.47:52494) 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9755d22700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x56452868a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756523700 1 -- 172.21.15.47:0/3425889818 learned_addr learned my addr 172.21.15.47:0/3425889818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756523700 1 -- 172.21.15.47:0/3425889818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 msgr2=0x56452868a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756523700 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x56452868a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756523700 1 -- 172.21.15.47:0/3425889818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56452868d660 msgr2=0x56452868a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756523700 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56452868d660 0x56452868a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.457+0000 7f9756523700 1 -- 172.21.15.47:0/3425889818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56452851a9d0 con 0x564528681e60 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9756523700 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528681e60 0x56452868cd30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f974c009770 tx=0x7f974c003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9189aa5183f550e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9739ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9739ffb700 1 -- 172.21.15.47:0/3425889818 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c026c70 con 0x564528681e60 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9739ffb700 1 -- 172.21.15.47:0/3425889818 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f974c005050 con 0x564528681e60 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9763010b80 1 -- 172.21.15.47:0/3425889818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f974c0268b0 con 0x564528681e60 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9739ffb700 1 -- 172.21.15.47:0/3425889818 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f974c005250 con 0x564528681e60 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9763010b80 10 client.25253 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9763010b80 1 -- 172.21.15.47:0/3425889818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56452869a980 con 0x564528681e60 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9739ffb700 1 -- 172.21.15.47:0/3425889818 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974c0268b0 con 0x564528681e60 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9739ffb700 1 -- 172.21.15.47:0/3425889818 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f974c0324d0 con 0x564528681e60 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9763010b80 10 client.25253 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9763010b80 10 client.25253 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.458+0000 7f9763010b80 20 client.25253 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.459+0000 7f9763010b80 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56452869ceb0 0x56452869f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.459+0000 7f9763010b80 4 client.25253 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.459+0000 7f9763010b80 1 -- 172.21.15.47:0/3425889818 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645285da580 con 0x56452869ceb0 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.459+0000 7f9755d22700 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56452869ceb0 0x56452869f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.459+0000 7f9755d22700 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56452869ceb0 0x56452869f2a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c28d09986b68e0 server_cookie=fe240896e6d39778 in_seq=0 out_seq=0 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.459+0000 7f9739ffb700 10 client.25253 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9739ffb700 1 -- 172.21.15.47:0/3425889818 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645285da580 con 0x56452869ceb0 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9739ffb700 10 client.25253 handle_command_reply: tid=0 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 client.25253 shutdown 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 -- 172.21.15.47:0/3425889818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528681e60 msgr2=0x56452868cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528681e60 0x56452868cd30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f974c009770 tx=0x7f974c003d40 comp rx=0 tx=0).stop 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 -- 172.21.15.47:0/3425889818 shutdown_connections 2024-04-25T21:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56452869ceb0 0x56452869f2a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56452868d660 0x56452868a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9739ffb700 0 client.25253 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528681e60 0x56452868cd30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 --2- 172.21.15.47:0/3425889818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x56452868a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 -- 172.21.15.47:0/3425889818 >> 172.21.15.47:0/3425889818 conn(0x5645285a6990 msgr2=0x564528674c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 -- 172.21.15.47:0/3425889818 shutdown_connections 2024-04-25T21:02:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 1 -- 172.21.15.47:0/3425889818 wait complete. 2024-04-25T21:02:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.460+0000 7f9763010b80 20 client.25253 trim_cache size 0 max 16384 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.480+0000 7f9763010b80 1 Processor -- start 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.480+0000 7f9763010b80 1 -- start start 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.480+0000 7f9763010b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528684490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.480+0000 7f9763010b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528682780 0x5645286849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.480+0000 7f9763010b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56452868d660 0x5645286a4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.480+0000 7f9763010b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f97501ad390 con 0x564528681e60 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.480+0000 7f9763010b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97501ada60 con 0x564528682780 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.480+0000 7f9763010b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f97501ab290 con 0x56452868d660 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528684490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528684490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34322/0 (socket says 172.21.15.47:34322) 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 -- 172.21.15.47:0/1384887711 learned_addr learned my addr 172.21.15.47:0/1384887711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9755d22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528682780 0x5645286849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756d24700 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56452868d660 0x5645286a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 -- 172.21.15.47:0/1384887711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56452868d660 msgr2=0x5645286a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56452868d660 0x5645286a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 -- 172.21.15.47:0/1384887711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528682780 msgr2=0x5645286849d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528682780 0x5645286849d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 -- 172.21.15.47:0/1384887711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564528682b60 con 0x564528681e60 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9755d22700 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528682780 0x5645286849d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756d24700 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56452868d660 0x5645286a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f9756523700 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528684490 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f974c009e70 tx=0x7f974c009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69e76ec80abfd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f973bfff700 1 -- 172.21.15.47:0/1384887711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c032c40 con 0x564528681e60 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f973bfff700 1 -- 172.21.15.47:0/1384887711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974c003db0 con 0x564528681e60 2024-04-25T21:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.481+0000 7f973bfff700 1 -- 172.21.15.47:0/1384887711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c0046d0 con 0x564528681e60 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 -- 172.21.15.47:0/1384887711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 msgr2=0x564528684490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528684490 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f974c009e70 tx=0x7f974c009350 comp rx=0 tx=0).stop 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 -- 172.21.15.47:0/1384887711 shutdown_connections 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528684490 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56452868d660 0x5645286a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 --2- 172.21.15.47:0/1384887711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564528682780 0x5645286849d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 -- 172.21.15.47:0/1384887711 >> 172.21.15.47:0/1384887711 conn(0x5645285a6990 msgr2=0x564528683d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 -- 172.21.15.47:0/1384887711 shutdown_connections 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 -- 172.21.15.47:0/1384887711 wait complete. 2024-04-25T21:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 -- start start 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.482+0000 7f9763010b80 1 Processor -- start 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9763010b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528695940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9763010b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x564528690550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9763010b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x564528690a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9763010b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f97501a9750 con 0x564528681e60 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9763010b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97501aabc0 con 0x56452868d660 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9763010b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f97501a9e20 con 0x564528682780 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9763010b80 10 client.0 _ll_register_callbacks cb 0x564528565a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9756523700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528695940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x564528690550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9756d24700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x564528690a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x564528690550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:33786/0 (socket says 172.21.15.47:33786) 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 -- 172.21.15.47:0/1064341333 learned_addr learned my addr 172.21.15.47:0/1064341333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9756d24700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x564528690a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:52520/0 (socket says 172.21.15.47:52520) 2024-04-25T21:02:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 -- 172.21.15.47:0/1064341333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 msgr2=0x564528690a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x564528690a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 -- 172.21.15.47:0/1064341333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 msgr2=0x564528695940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528695940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 -- 172.21.15.47:0/1064341333 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645286912d0 con 0x564528682780 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9755d22700 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x564528690550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f974000c5b0 tx=0x7f97400bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b218ee3d3d211a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9739ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.483+0000 7f9739ffb700 1 -- 172.21.15.47:0/1064341333 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97400b5070 con 0x564528682780 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 1 -- 172.21.15.47:0/1064341333 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f974c003db0 con 0x564528682780 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9739ffb700 1 -- 172.21.15.47:0/1064341333 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f974009cbb0 con 0x564528682780 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9739ffb700 1 -- 172.21.15.47:0/1064341333 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97400a55c0 con 0x564528682780 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 10 client.25252 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 1 -- 172.21.15.47:0/1064341333 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564528704180 con 0x564528682780 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9739ffb700 1 -- 172.21.15.47:0/1064341333 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974c003db0 con 0x564528682780 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9739ffb700 1 -- 172.21.15.47:0/1064341333 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f97400870b0 con 0x564528682780 2024-04-25T21:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 10 client.25252 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 10 client.25252 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 20 client.25252 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564528704870 0x564528706c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 4 client.25252 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9ebddfb8", "await_for": 42.0}] 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9763010b80 1 -- 172.21.15.47:0/1064341333 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9ebddfb8", "await_for": 42.0}) -- 0x5645285d4620 con 0x564528704870 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.484+0000 7f9756523700 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564528704870 0x564528706c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9756523700 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564528704870 0x564528706c90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752329835a6a0ae9 server_cookie=3a3ba7d15c1f7617 in_seq=0 out_seq=0 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9739ffb700 10 client.25252 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9739ffb700 1 -- 172.21.15.47:0/1064341333 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: -110 ) ==== 8+0+590 (crc 0 0 0) 0x5645285d4620 con 0x564528704870 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9739ffb700 10 client.25252 handle_command_reply: tid=0 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 client.25252 shutdown 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 -- 172.21.15.47:0/1064341333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 msgr2=0x564528690550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x564528690550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f974000c5b0 tx=0x7f97400bd040 comp rx=0 tx=0).stop 2024-04-25T21:02:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 -- 172.21.15.47:0/1064341333 shutdown_connections 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564528704870 0x564528706c90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564528681e60 0x564528695940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452868d660 0x564528690a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9739ffb700 0 client.25252 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 --2- 172.21.15.47:0/1064341333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564528682780 0x564528690550 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 -- 172.21.15.47:0/1064341333 >> 172.21.15.47:0/1064341333 conn(0x5645285a6990 msgr2=0x564528684610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 -- 172.21.15.47:0/1064341333 shutdown_connections 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 1 -- 172.21.15.47:0/1064341333 wait complete. 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:02:42.485+0000 7f9763010b80 20 client.25252 trim_cache size 0 max 16384 2024-04-25T21:02:42.491 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 30, 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "leader": 24488, 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "set_version": 6, 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "sets": { 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "9ebddfb8": { 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "version": 6, 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "age_ref": 7.3, 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "state": { 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "TIMEDOUT", 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.0 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "timeout": 60.0, 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "expiration": 86.2, 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "members": { 2024-04-25T21:02:42.492 INFO:teuthology.orchestra.run.smithi047.stdout: "file:/": { 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: "excluded": false, 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: "state": { 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "QUIESCING", 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 60.0 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:02:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:Error ETIMEDOUT: 2024-04-25T21:02:42.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:42 smithi092 ceph-mon[32524]: pgmap v84: 97 pgs: 97 active+clean; 21 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 202 KiB/s wr, 42 op/s 2024-04-25T21:02:42.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:42 smithi186 ceph-mon[32529]: pgmap v84: 97 pgs: 97 active+clean; 21 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 202 KiB/s wr, 42 op/s 2024-04-25T21:02:43.235 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-25T21:02:43.235 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: 110 (ETIMEDOUT), stdout: { "epoch": 30, "leader": 24488, "set_version": 6, "sets": { "9ebddfb8": { "version": 6, "age_ref": 7.3, "state": { "name": "TIMEDOUT", "age": 0.0 }, "timeout": 60.0, "expiration": 86.2, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCING", "age": 60.0 } } } } } } 2024-04-25T21:02:43.235 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks/quiescer.py", line 353, in _run self.do_quiesce(d) File "/home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks/quiescer.py", line 308, in do_quiesce raise RuntimeError(f"Error quiescing set '{set_id}': {rcinfo}") RuntimeError: Error quiescing set '9ebddfb8': 110 (ETIMEDOUT) 2024-04-25T21:02:43.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:43 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:43.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:43 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:43.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:43 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:44.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:44 smithi047 ceph-mon[29195]: pgmap v85: 97 pgs: 97 active+clean; 21 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 202 KiB/s wr, 39 op/s 2024-04-25T21:02:44.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:44 smithi186 ceph-mon[32529]: pgmap v85: 97 pgs: 97 active+clean; 21 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 202 KiB/s wr, 39 op/s 2024-04-25T21:02:44.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:44 smithi092 ceph-mon[32524]: pgmap v85: 97 pgs: 97 active+clean; 21 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 202 KiB/s wr, 39 op/s 2024-04-25T21:02:46.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:46 smithi047 ceph-mon[29195]: pgmap v86: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 658 KiB/s wr, 60 op/s 2024-04-25T21:02:46.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:46 smithi186 ceph-mon[32529]: pgmap v86: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 658 KiB/s wr, 60 op/s 2024-04-25T21:02:46.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:46 smithi092 ceph-mon[32524]: pgmap v86: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 658 KiB/s wr, 60 op/s 2024-04-25T21:02:48.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:48 smithi047 ceph-mon[29195]: pgmap v87: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 656 KiB/s wr, 57 op/s 2024-04-25T21:02:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 -- 172.21.15.47:0/201306060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380102b70 msgr2=0x7ff380106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 --2- 172.21.15.47:0/201306060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380102b70 0x7ff380106bb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff374009ca0 tx=0x7ff37402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 -- 172.21.15.47:0/201306060 shutdown_connections 2024-04-25T21:02:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 --2- 172.21.15.47:0/201306060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff380107290 0x7ff38010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 --2- 172.21.15.47:0/201306060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380102b70 0x7ff380106bb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 --2- 172.21.15.47:0/201306060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3801021c0 0x7ff3801025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 -- 172.21.15.47:0/201306060 >> 172.21.15.47:0/201306060 conn(0x7ff3800fda20 msgr2=0x7ff3800ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 -- 172.21.15.47:0/201306060 shutdown_connections 2024-04-25T21:02:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.648+0000 7ff384ebb640 1 -- 172.21.15.47:0/201306060 wait complete. 2024-04-25T21:02:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff384ebb640 1 Processor -- start 2024-04-25T21:02:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff384ebb640 1 -- start start 2024-04-25T21:02:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff384ebb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3801021c0 0x7ff380074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff384ebb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff380107290 0x7ff380074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff384ebb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380074cd0 0x7ff3801a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff384ebb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801121d0 con 0x7ff380074cd0 2024-04-25T21:02:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff384ebb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff380112050 con 0x7ff3801021c0 2024-04-25T21:02:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff384ebb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff380112350 con 0x7ff380107290 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380074cd0 0x7ff3801a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380074cd0 0x7ff3801a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35314/0 (socket says 172.21.15.47:35314) 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37ed9d640 1 -- 172.21.15.47:0/3228190841 learned_addr learned my addr 172.21.15.47:0/3228190841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff380107290 0x7ff380074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37e59c640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3801021c0 0x7ff380074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37ed9d640 1 -- 172.21.15.47:0/3228190841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff380107290 msgr2=0x7ff380074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37ed9d640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff380107290 0x7ff380074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37ed9d640 1 -- 172.21.15.47:0/3228190841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3801021c0 msgr2=0x7ff380074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37ed9d640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3801021c0 0x7ff380074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37ed9d640 1 -- 172.21.15.47:0/3228190841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3801a6fc0 con 0x7ff380074cd0 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff380107290 0x7ff380074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.649+0000 7ff37e59c640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3801021c0 0x7ff380074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:02:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.650+0000 7ff37ed9d640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380074cd0 0x7ff3801a68c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff37800d950 tx=0x7ff37800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.650+0000 7ff36f7fe640 1 -- 172.21.15.47:0/3228190841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378017030 con 0x7ff380074cd0 2024-04-25T21:02:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.650+0000 7ff384ebb640 1 -- 172.21.15.47:0/3228190841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3801a72b0 con 0x7ff380074cd0 2024-04-25T21:02:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.650+0000 7ff36f7fe640 1 -- 172.21.15.47:0/3228190841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff378004020 con 0x7ff380074cd0 2024-04-25T21:02:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.650+0000 7ff36f7fe640 1 -- 172.21.15.47:0/3228190841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378003b50 con 0x7ff380074cd0 2024-04-25T21:02:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.650+0000 7ff384ebb640 1 -- 172.21.15.47:0/3228190841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3801a77b0 con 0x7ff380074cd0 2024-04-25T21:02:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.652+0000 7ff36f7fe640 1 -- 172.21.15.47:0/3228190841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff37801a070 con 0x7ff380074cd0 2024-04-25T21:02:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.652+0000 7ff387bcf740 1 Processor -- start 2024-04-25T21:02:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff387bcf740 1 -- start start 2024-04-25T21:02:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0620 0x55d7037e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037edbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff387bcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d703745970 con 0x55d7037eb800 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff387bcf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d7036e24a0 con 0x55d7037e0620 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff387bcf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d703741930 con 0x55d7037e0f40 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff36f7fe640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff3680a3df0 0x7ff3680a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037edbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037edbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35326/0 (socket says 172.21.15.47:35326) 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 -- 172.21.15.47:0/2439614918 learned_addr learned my addr 172.21.15.47:0/2439614918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0620 0x55d7037e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37e59c640 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37ed9d640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff3680a3df0 0x7ff3680a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff36f7fe640 1 -- 172.21.15.47:0/3228190841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ff3780cfff0 con 0x7ff380074cd0 2024-04-25T21:02:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 -- 172.21.15.47:0/2439614918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 msgr2=0x55d7037eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 -- 172.21.15.47:0/2439614918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0620 msgr2=0x55d7037e0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0620 0x55d7037e0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37dd9b640 1 -- 172.21.15.47:0/2439614918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d703458210 con 0x55d7037eb800 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.653+0000 7ff37e59c640 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.654+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037edbf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff374004af0 tx=0x7ff374005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea37d1f445f146 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.654+0000 7ff37ed9d640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff3680a3df0 0x7ff3680a62b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3780cdd50 tx=0x7ff378086ad0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.654+0000 7ff36cff9640 1 -- 172.21.15.47:0/2439614918 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374032ed0 con 0x55d7037eb800 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.654+0000 7ff36cff9640 1 -- 172.21.15.47:0/2439614918 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff374036040 con 0x55d7037eb800 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.654+0000 7ff36cff9640 1 -- 172.21.15.47:0/2439614918 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004040 con 0x55d7037eb800 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- 172.21.15.47:0/2439614918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 msgr2=0x55d7037edbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037edbf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff374004af0 tx=0x7ff374005d80 comp rx=0 tx=0).stop 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- 172.21.15.47:0/2439614918 shutdown_connections 2024-04-25T21:02:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037edbf0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 --2- 172.21.15.47:0/2439614918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0620 0x55d7037e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- 172.21.15.47:0/2439614918 >> 172.21.15.47:0/2439614918 conn(0x55d7037dcb30 msgr2=0x55d7037dcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- 172.21.15.47:0/2439614918 shutdown_connections 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- 172.21.15.47:0/2439614918 wait complete. 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- start start 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 Processor -- start 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0620 0x55d7037eb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037e8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 0x55d7037e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d7037ee250 con 0x55d7037e0620 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d7036e00b0 con 0x55d7037eb800 2024-04-25T21:02:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.655+0000 7ff387bcf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d7037ee3d0 con 0x55d7037e0f40 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff387bcf740 10 client.0 _ll_register_callbacks cb 0x55d7036cdd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0620 0x55d7037eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0620 0x55d7037eb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35330/0 (socket says 172.21.15.47:35330) 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 -- 172.21.15.47:0/1221327759 learned_addr learned my addr 172.21.15.47:0/1221327759 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37e59c640 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 -- 172.21.15.47:0/1221327759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 msgr2=0x55d7037e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037e8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 -- 172.21.15.47:0/1221327759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 msgr2=0x55d7037e89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 0x55d7037e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 -- 172.21.15.47:0/1221327759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d703524620 con 0x55d7037e0620 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0620 0x55d7037eb170 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff374006010 tx=0x7ff374003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fda9618b03a611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:02:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff365ffb640 1 -- 172.21.15.47:0/1221327759 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37404f020 con 0x55d7037e0620 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff365ffb640 1 -- 172.21.15.47:0/1221327759 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ff374031450 con 0x55d7037e0620 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff387bcf740 1 -- 172.21.15.47:0/1221327759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff374036040 con 0x55d7037e0620 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff365ffb640 1 -- 172.21.15.47:0/1221327759 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff374031650 con 0x55d7037e0620 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff387bcf740 10 client.15645 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.656+0000 7ff387bcf740 1 -- 172.21.15.47:0/1221327759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7037ee250 con 0x55d7037e0620 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff365ffb640 1 -- 172.21.15.47:0/1221327759 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff374036040 con 0x55d7037e0620 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff365ffb640 1 -- 172.21.15.47:0/1221327759 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff374003f50 con 0x55d7037e0620 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff387bcf740 10 client.15645 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff387bcf740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff387bcf740 20 client.15645 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55d7037fb350 0x55d7037fd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff387bcf740 4 client.15645 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff387bcf740 1 -- 172.21.15.47:0/1221327759 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d703753390 con 0x55d7037fb350 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.657+0000 7ff37e59c640 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55d7037fb350 0x55d7037fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff37e59c640 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55d7037fb350 0x55d7037fd740 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3afde4d2f97cddcd server_cookie=44cc7d370730a411 in_seq=0 out_seq=0 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff365ffb640 10 client.15645 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff365ffb640 1 -- 172.21.15.47:0/1221327759 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d703753390 con 0x55d7037fb350 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff365ffb640 10 client.15645 handle_command_reply: tid=0 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff387bcf740 1 client.15645 shutdown 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff387bcf740 1 -- 172.21.15.47:0/1221327759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0620 msgr2=0x55d7037eb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0620 0x55d7037eb170 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff374006010 tx=0x7ff374003ba0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff387bcf740 1 -- 172.21.15.47:0/1221327759 shutdown_connections 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.658+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55d7037fb350 0x55d7037fd740 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.659+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 0x55d7037e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.659+0000 7ff365ffb640 0 client.15645 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.659+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0f40 0x55d7037e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.659+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1221327759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0620 0x55d7037eb170 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.659+0000 7ff387bcf740 1 -- 172.21.15.47:0/1221327759 >> 172.21.15.47:0/1221327759 conn(0x55d7037dcb30 msgr2=0x55d7037e2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.659+0000 7ff387bcf740 1 -- 172.21.15.47:0/1221327759 shutdown_connections 2024-04-25T21:02:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.659+0000 7ff387bcf740 1 -- 172.21.15.47:0/1221327759 wait complete. 2024-04-25T21:02:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.659+0000 7ff387bcf740 20 client.15645 trim_cache size 0 max 16384 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff387bcf740 1 Processor -- start 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff387bcf740 1 -- start start 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 0x55d7037e8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0f40 0x55d7037eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 0x55d703802ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff387bcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801b2600 con 0x55d7037e0f40 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff387bcf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801afe70 con 0x55d7037eb800 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff387bcf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801adce0 con 0x55d7037e0620 2024-04-25T21:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff37e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0f40 0x55d7037eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff37e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0f40 0x55d7037eaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35336/0 (socket says 172.21.15.47:35336) 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.674+0000 7ff37e59c640 1 -- 172.21.15.47:0/2864146724 learned_addr learned my addr 172.21.15.47:0/2864146724 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 0x55d703802ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff37e59c640 1 -- 172.21.15.47:0/2864146724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 msgr2=0x55d7037e8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff37e59c640 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 0x55d7037e8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff37e59c640 1 -- 172.21.15.47:0/2864146724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 msgr2=0x55d703802ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff37e59c640 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 0x55d703802ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff37e59c640 1 -- 172.21.15.47:0/2864146724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7037fdab0 con 0x55d7037e0f40 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 0x55d703802ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff37e59c640 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0f40 0x55d7037eaca0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55d7037e1bc0 tx=0x7ff3700138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51a006287ba411c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff367fff640 1 -- 172.21.15.47:0/2864146724 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37001d430 con 0x55d7037e0f40 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff367fff640 1 -- 172.21.15.47:0/2864146724 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff370005020 con 0x55d7037e0f40 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff367fff640 1 -- 172.21.15.47:0/2864146724 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370004c40 con 0x55d7037e0f40 2024-04-25T21:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff387bcf740 1 -- 172.21.15.47:0/2864146724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0f40 msgr2=0x55d7037eaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.675+0000 7ff387bcf740 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0f40 0x55d7037eaca0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55d7037e1bc0 tx=0x7ff3700138f0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 -- 172.21.15.47:0/2864146724 shutdown_connections 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037eb800 0x55d703802ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037e0f40 0x55d7037eaca0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 --2- 172.21.15.47:0/2864146724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 0x55d7037e8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 -- 172.21.15.47:0/2864146724 >> 172.21.15.47:0/2864146724 conn(0x55d7037dcb30 msgr2=0x55d7037e7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 -- 172.21.15.47:0/2864146724 shutdown_connections 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 -- 172.21.15.47:0/2864146724 wait complete. 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 -- start start 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 Processor -- start 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 0x55d7037f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0f40 0x55d7037f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801ae330 con 0x55d7037eb800 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801af7a0 con 0x55d7037e0f40 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801aea00 con 0x55d7037e0620 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff387bcf740 10 client.0 _ll_register_callbacks cb 0x7ff374032da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037f0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35348/0 (socket says 172.21.15.47:35348) 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff37dd9b640 1 -- 172.21.15.47:0/1959861561 learned_addr learned my addr 172.21.15.47:0/1959861561 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff37e59c640 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0f40 0x55d7037f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.676+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 0x55d7037f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff37dd9b640 1 -- 172.21.15.47:0/1959861561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 msgr2=0x55d7037f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 0x55d7037f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff37dd9b640 1 -- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0f40 msgr2=0x55d7037f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0f40 0x55d7037f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff37dd9b640 1 -- 172.21.15.47:0/1959861561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7037e0d70 con 0x55d7037eb800 2024-04-25T21:02:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff37e59c640 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0f40 0x55d7037f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff37dd9b640 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037f0a30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff3740049d0 tx=0x7ff374004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3652540dd4d03111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:02:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff365ffb640 1 -- 172.21.15.47:0/1959861561 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374047040 con 0x55d7037eb800 2024-04-25T21:02:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff365ffb640 1 -- 172.21.15.47:0/1959861561 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ff37402f390 con 0x55d7037eb800 2024-04-25T21:02:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff387bcf740 1 -- 172.21.15.47:0/1959861561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff370005020 con 0x55d7037eb800 2024-04-25T21:02:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff365ffb640 1 -- 172.21.15.47:0/1959861561 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff37404f040 con 0x55d7037eb800 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff387bcf740 10 client.15651 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff387bcf740 1 -- 172.21.15.47:0/1959861561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d703861210 con 0x55d7037eb800 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.677+0000 7ff365ffb640 1 -- 172.21.15.47:0/1959861561 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff370005020 con 0x55d7037eb800 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff365ffb640 1 -- 172.21.15.47:0/1959861561 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff3740305c0 con 0x55d7037eb800 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff387bcf740 10 client.15651 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff387bcf740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff387bcf740 20 client.15651 populate_metadata read hostname 'smithi047' 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55d7037fb350 0x55d7038637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff387bcf740 4 client.15651 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff387bcf740 1 -- 172.21.15.47:0/1959861561 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d70374d200 con 0x55d7037fb350 2024-04-25T21:02:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff37e59c640 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55d7037fb350 0x55d7038637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff37e59c640 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55d7037fb350 0x55d7038637b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b542d62018e70c5e server_cookie=8d33887a2fbe3da5 in_seq=0 out_seq=0 2024-04-25T21:02:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.678+0000 7ff365ffb640 10 client.15651 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.688+0000 7ff365ffb640 1 -- 172.21.15.47:0/1959861561 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d70374d200 con 0x55d7037fb350 2024-04-25T21:02:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.688+0000 7ff365ffb640 10 client.15651 handle_command_reply: tid=0 2024-04-25T21:02:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.688+0000 7ff387bcf740 1 client.15651 shutdown 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 -- 172.21.15.47:0/1959861561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 msgr2=0x55d7037f0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037f0a30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff3740049d0 tx=0x7ff374004e50 comp rx=0 tx=0).stop 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 -- 172.21.15.47:0/1959861561 shutdown_connections 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55d7037fb350 0x55d7038637b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7037eb800 0x55d7037f0a30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7037e0f40 0x55d7037f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff365ffb640 0 client.15651 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 --2- 172.21.15.47:0/1959861561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d7037e0620 0x55d7037f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 -- 172.21.15.47:0/1959861561 >> 172.21.15.47:0/1959861561 conn(0x55d7037dcb30 msgr2=0x55d7037db420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 -- 172.21.15.47:0/1959861561 shutdown_connections 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 1 -- 172.21.15.47:0/1959861561 wait complete. 2024-04-25T21:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff387bcf740 20 client.15651 trim_cache size 0 max 16384 2024-04-25T21:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 -- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff3680a3df0 msgr2=0x7ff3680a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff3680a3df0 0x7ff3680a62b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3780cdd50 tx=0x7ff378086ad0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 -- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380074cd0 msgr2=0x7ff3801a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380074cd0 0x7ff3801a68c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff37800d950 tx=0x7ff37800de10 comp rx=0 tx=0).stop 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 -- 172.21.15.47:0/3228190841 shutdown_connections 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff380074cd0 0x7ff3801a68c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff380107290 0x7ff380074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff3680a3df0 0x7ff3680a62b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.689+0000 7ff384ebb640 1 --2- 172.21.15.47:0/3228190841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3801021c0 0x7ff380074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.690+0000 7ff384ebb640 1 -- 172.21.15.47:0/3228190841 >> 172.21.15.47:0/3228190841 conn(0x7ff3800fda20 msgr2=0x7ff3800ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.690+0000 7ff384ebb640 1 -- 172.21.15.47:0/3228190841 shutdown_connections 2024-04-25T21:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:02:48.690+0000 7ff384ebb640 1 -- 172.21.15.47:0/3228190841 wait complete. 2024-04-25T21:02:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:48 smithi186 ceph-mon[32529]: pgmap v87: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 656 KiB/s wr, 57 op/s 2024-04-25T21:02:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:48 smithi092 ceph-mon[32524]: pgmap v87: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 656 KiB/s wr, 57 op/s 2024-04-25T21:02:50.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:50 smithi047 ceph-mon[29195]: pgmap v88: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 656 KiB/s wr, 57 op/s 2024-04-25T21:02:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:50 smithi186 ceph-mon[32529]: pgmap v88: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 656 KiB/s wr, 57 op/s 2024-04-25T21:02:50.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:50 smithi092 ceph-mon[32524]: pgmap v88: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 656 KiB/s wr, 57 op/s 2024-04-25T21:02:52.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:52 smithi047 ceph-mon[29195]: pgmap v89: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.6 MiB/s wr, 273 op/s 2024-04-25T21:02:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:52 smithi186 ceph-mon[32529]: pgmap v89: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.6 MiB/s wr, 273 op/s 2024-04-25T21:02:52.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:52 smithi092 ceph-mon[32524]: pgmap v89: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.6 MiB/s wr, 273 op/s 2024-04-25T21:02:53.543 INFO:tasks.workunit.client.0.smithi047.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 8% (359/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 11% (472/4290) Updating files: 12% (515/4290) Updating files: 12% (517/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 48% (2071/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 61% (2653/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 68% (2946/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 72% (3111/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 89% (3853/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-25T21:02:53.555 INFO:tasks.workunit.client.0.smithi047.stderr:+ cd xfstests-dev 2024-04-25T21:02:53.556 INFO:tasks.workunit.client.0.smithi047.stderr:+ make -j4 2024-04-25T21:02:53.593 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize -cfi 2024-04-25T21:02:54.018 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-25T21:02:54.018 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file './config.guess' 2024-04-25T21:02:54.023 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file './config.sub' 2024-04-25T21:02:54.028 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file './install-sh' 2024-04-25T21:02:54.033 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file './ltmain.sh' 2024-04-25T21:02:54.061 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-25T21:02:54.062 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-25T21:02:54.068 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-25T21:02:54.075 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-25T21:02:54.081 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-25T21:02:54.087 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-25T21:02:54.093 INFO:tasks.workunit.client.0.smithi047.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-25T21:02:54.094 INFO:tasks.workunit.client.0.smithi047.stdout:cp include/install-sh . 2024-04-25T21:02:54.108 INFO:tasks.workunit.client.0.smithi047.stdout:aclocal -I m4 2024-04-25T21:02:54.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:54 smithi047 ceph-mon[29195]: pgmap v90: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-04-25T21:02:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:54 smithi186 ceph-mon[32529]: pgmap v90: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-04-25T21:02:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:54 smithi092 ceph-mon[32524]: pgmap v90: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-04-25T21:02:55.155 INFO:tasks.workunit.client.0.smithi047.stdout:autoheader 2024-04-25T21:02:55.488 INFO:tasks.workunit.client.0.smithi047.stdout:autoconf 2024-04-25T21:02:55.674 INFO:tasks.workunit.client.0.smithi047.stdout:./configure \ 2024-04-25T21:02:55.674 INFO:tasks.workunit.client.0.smithi047.stdout: --libexecdir=/usr/lib \ 2024-04-25T21:02:55.674 INFO:tasks.workunit.client.0.smithi047.stdout: --exec_prefix=/var/lib 2024-04-25T21:02:55.815 INFO:tasks.workunit.client.0.smithi047.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-25T21:02:55.815 INFO:tasks.workunit.client.0.smithi047.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-25T21:02:55.816 INFO:tasks.workunit.client.0.smithi047.stdout:checking how to print strings... printf 2024-04-25T21:02:55.816 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gcc... gcc 2024-04-25T21:02:55.907 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether the C compiler works... yes 2024-04-25T21:02:55.907 INFO:tasks.workunit.client.0.smithi047.stdout:checking for C compiler default output file name... a.out 2024-04-25T21:02:55.946 INFO:tasks.workunit.client.0.smithi047.stdout:checking for suffix of executables... 2024-04-25T21:02:55.983 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether we are cross compiling... no 2024-04-25T21:02:56.021 INFO:tasks.workunit.client.0.smithi047.stdout:checking for suffix of object files... o 2024-04-25T21:02:56.045 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T21:02:56.070 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether gcc accepts -g... yes 2024-04-25T21:02:56.165 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T21:02:56.169 INFO:tasks.workunit.client.0.smithi047.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-25T21:02:56.173 INFO:tasks.workunit.client.0.smithi047.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T21:02:56.174 INFO:tasks.workunit.client.0.smithi047.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T21:02:56.175 INFO:tasks.workunit.client.0.smithi047.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-25T21:02:56.179 INFO:tasks.workunit.client.0.smithi047.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-25T21:02:56.180 INFO:tasks.workunit.client.0.smithi047.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-25T21:02:56.190 INFO:tasks.workunit.client.0.smithi047.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-25T21:02:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:56 smithi047 ceph-mon[29195]: pgmap v91: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-04-25T21:02:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:56 smithi186 ceph-mon[32529]: pgmap v91: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-04-25T21:02:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:56 smithi092 ceph-mon[32524]: pgmap v91: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-04-25T21:02:56.999 INFO:tasks.workunit.client.0.smithi047.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-25T21:02:56.999 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether ln -s works... yes 2024-04-25T21:02:57.012 INFO:tasks.workunit.client.0.smithi047.stdout:checking the maximum length of command line arguments... 1572864 2024-04-25T21:02:57.017 INFO:tasks.workunit.client.0.smithi047.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-25T21:02:57.017 INFO:tasks.workunit.client.0.smithi047.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-25T21:02:57.017 INFO:tasks.workunit.client.0.smithi047.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-25T21:02:57.017 INFO:tasks.workunit.client.0.smithi047.stdout:checking for objdump... objdump 2024-04-25T21:02:57.017 INFO:tasks.workunit.client.0.smithi047.stdout:checking how to recognize dependent libraries... pass_all 2024-04-25T21:02:57.018 INFO:tasks.workunit.client.0.smithi047.stdout:checking for dlltool... no 2024-04-25T21:02:57.018 INFO:tasks.workunit.client.0.smithi047.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-25T21:02:57.018 INFO:tasks.workunit.client.0.smithi047.stdout:checking for ar... ar 2024-04-25T21:02:57.119 INFO:tasks.workunit.client.0.smithi047.stdout:checking for archiver @FILE support... @ 2024-04-25T21:02:57.119 INFO:tasks.workunit.client.0.smithi047.stdout:checking for strip... strip 2024-04-25T21:02:57.120 INFO:tasks.workunit.client.0.smithi047.stdout:checking for ranlib... ranlib 2024-04-25T21:02:57.120 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gawk... gawk 2024-04-25T21:02:57.231 INFO:tasks.workunit.client.0.smithi047.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-25T21:02:57.232 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sysroot... no 2024-04-25T21:02:57.243 INFO:tasks.workunit.client.0.smithi047.stdout:checking for a working dd... /usr/bin/dd 2024-04-25T21:02:57.255 INFO:tasks.workunit.client.0.smithi047.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-25T21:02:57.297 INFO:tasks.workunit.client.0.smithi047.stdout:checking for mt... no 2024-04-25T21:02:57.312 INFO:tasks.workunit.client.0.smithi047.stdout:checking if : is a manifest tool... no 2024-04-25T21:02:57.422 INFO:tasks.workunit.client.0.smithi047.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T21:02:57.633 INFO:tasks.workunit.client.0.smithi047.stdout:checking for ANSI C header files... yes 2024-04-25T21:02:57.668 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/types.h... yes 2024-04-25T21:02:57.707 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/stat.h... yes 2024-04-25T21:02:57.746 INFO:tasks.workunit.client.0.smithi047.stdout:checking for stdlib.h... yes 2024-04-25T21:02:57.785 INFO:tasks.workunit.client.0.smithi047.stdout:checking for string.h... yes 2024-04-25T21:02:57.826 INFO:tasks.workunit.client.0.smithi047.stdout:checking for memory.h... yes 2024-04-25T21:02:57.866 INFO:tasks.workunit.client.0.smithi047.stdout:checking for strings.h... yes 2024-04-25T21:02:57.906 INFO:tasks.workunit.client.0.smithi047.stdout:checking for inttypes.h... yes 2024-04-25T21:02:57.946 INFO:tasks.workunit.client.0.smithi047.stdout:checking for stdint.h... yes 2024-04-25T21:02:57.989 INFO:tasks.workunit.client.0.smithi047.stdout:checking for unistd.h... yes 2024-04-25T21:02:58.034 INFO:tasks.workunit.client.0.smithi047.stdout:checking for dlfcn.h... yes 2024-04-25T21:02:58.040 INFO:tasks.workunit.client.0.smithi047.stdout:checking for objdir... .libs 2024-04-25T21:02:58.155 INFO:tasks.workunit.client.0.smithi047.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-25T21:02:58.155 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-25T21:02:58.193 INFO:tasks.workunit.client.0.smithi047.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-25T21:02:58.247 INFO:tasks.workunit.client.0.smithi047.stdout:checking if gcc static flag -static works... no 2024-04-25T21:02:58.314 INFO:tasks.workunit.client.0.smithi047.stdout:checking if gcc supports -c -o file.o... yes 2024-04-25T21:02:58.314 INFO:tasks.workunit.client.0.smithi047.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-25T21:02:58.320 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-25T21:02:58.380 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether -lc should be explicitly linked in... no 2024-04-25T21:02:58.441 INFO:tasks.workunit.client.0.smithi047.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-25T21:02:58.441 INFO:tasks.workunit.client.0.smithi047.stdout:checking how to hardcode library paths into programs... immediate 2024-04-25T21:02:58.442 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether stripping libraries is possible... yes 2024-04-25T21:02:58.442 INFO:tasks.workunit.client.0.smithi047.stdout:checking if libtool supports shared libraries... yes 2024-04-25T21:02:58.442 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether to build shared libraries... yes 2024-04-25T21:02:58.442 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether to build static libraries... yes 2024-04-25T21:02:58.449 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gcc... (cached) gcc 2024-04-25T21:02:58.499 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T21:02:58.499 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T21:02:58.499 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T21:02:58.499 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gmake... /usr/bin/gmake 2024-04-25T21:02:58.499 INFO:tasks.workunit.client.0.smithi047.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-25T21:02:58.500 INFO:tasks.workunit.client.0.smithi047.stdout:checking for tar... /bin/tar 2024-04-25T21:02:58.500 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gzip... /bin/gzip 2024-04-25T21:02:58.526 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether gcc -MM is supported... yes 2024-04-25T21:02:58.526 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sort... /bin/sort 2024-04-25T21:02:58.526 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether ln -s works... yes 2024-04-25T21:02:58.526 INFO:tasks.workunit.client.0.smithi047.stdout:checking for rpm... /bin/rpm 2024-04-25T21:02:58.529 INFO:tasks.workunit.client.0.smithi047.stdout:checking for rpmbuild... no 2024-04-25T21:02:58.586 INFO:tasks.workunit.client.0.smithi047.stdout:checking assert.h usability... yes 2024-04-25T21:02:58.603 INFO:tasks.workunit.client.0.smithi047.stdout:checking assert.h presence... yes 2024-04-25T21:02:58.603 INFO:tasks.workunit.client.0.smithi047.stdout:checking for assert.h... yes 2024-04-25T21:02:58.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:02:58 smithi047 ceph-mon[29195]: pgmap v92: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-04-25T21:02:58.656 INFO:tasks.workunit.client.0.smithi047.stdout:checking bstring.h usability... no 2024-04-25T21:02:58.687 INFO:tasks.workunit.client.0.smithi047.stdout:checking bstring.h presence... no 2024-04-25T21:02:58.687 INFO:tasks.workunit.client.0.smithi047.stdout:checking for bstring.h... no 2024-04-25T21:02:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:02:58 smithi186 ceph-mon[32529]: pgmap v92: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-04-25T21:02:58.729 INFO:tasks.workunit.client.0.smithi047.stdout:checking libgen.h usability... yes 2024-04-25T21:02:58.745 INFO:tasks.workunit.client.0.smithi047.stdout:checking libgen.h presence... yes 2024-04-25T21:02:58.745 INFO:tasks.workunit.client.0.smithi047.stdout:checking for libgen.h... yes 2024-04-25T21:02:58.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:02:58 smithi092 ceph-mon[32524]: pgmap v92: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-04-25T21:02:58.804 INFO:tasks.workunit.client.0.smithi047.stdout:checking dirent.h usability... yes 2024-04-25T21:02:58.826 INFO:tasks.workunit.client.0.smithi047.stdout:checking dirent.h presence... yes 2024-04-25T21:02:58.826 INFO:tasks.workunit.client.0.smithi047.stdout:checking for dirent.h... yes 2024-04-25T21:02:58.875 INFO:tasks.workunit.client.0.smithi047.stdout:checking errno.h usability... yes 2024-04-25T21:02:58.894 INFO:tasks.workunit.client.0.smithi047.stdout:checking errno.h presence... yes 2024-04-25T21:02:58.894 INFO:tasks.workunit.client.0.smithi047.stdout:checking for errno.h... yes 2024-04-25T21:02:58.955 INFO:tasks.workunit.client.0.smithi047.stdout:checking malloc.h usability... yes 2024-04-25T21:02:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:checking malloc.h presence... yes 2024-04-25T21:02:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:checking for malloc.h... yes 2024-04-25T21:02:59.023 INFO:tasks.workunit.client.0.smithi047.stdout:checking uuid.h usability... no 2024-04-25T21:02:59.053 INFO:tasks.workunit.client.0.smithi047.stdout:checking uuid.h presence... no 2024-04-25T21:02:59.054 INFO:tasks.workunit.client.0.smithi047.stdout:checking for uuid.h... no 2024-04-25T21:02:59.102 INFO:tasks.workunit.client.0.smithi047.stdout:checking uuid/uuid.h usability... yes 2024-04-25T21:02:59.121 INFO:tasks.workunit.client.0.smithi047.stdout:checking uuid/uuid.h presence... yes 2024-04-25T21:02:59.121 INFO:tasks.workunit.client.0.smithi047.stdout:checking for uuid/uuid.h... yes 2024-04-25T21:02:59.177 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/uuid.h usability... no 2024-04-25T21:02:59.207 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/uuid.h presence... no 2024-04-25T21:02:59.207 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/uuid.h... no 2024-04-25T21:02:59.251 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/file.h usability... yes 2024-04-25T21:02:59.269 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/file.h presence... yes 2024-04-25T21:02:59.269 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/file.h... yes 2024-04-25T21:02:59.326 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/fcntl.h usability... yes 2024-04-25T21:02:59.344 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/fcntl.h presence... yes 2024-04-25T21:02:59.344 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/fcntl.h... yes 2024-04-25T21:02:59.395 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/syssgi.h usability... no 2024-04-25T21:02:59.424 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/syssgi.h presence... no 2024-04-25T21:02:59.424 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/syssgi.h... no 2024-04-25T21:02:59.496 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/param.h usability... yes 2024-04-25T21:02:59.518 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/param.h presence... yes 2024-04-25T21:02:59.518 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/param.h... yes 2024-04-25T21:02:59.531 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/stat.h... (cached) yes 2024-04-25T21:02:59.569 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/statvfs.h usability... yes 2024-04-25T21:02:59.586 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/statvfs.h presence... yes 2024-04-25T21:02:59.586 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/statvfs.h... yes 2024-04-25T21:02:59.633 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/time.h usability... yes 2024-04-25T21:02:59.650 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/time.h presence... yes 2024-04-25T21:02:59.650 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/time.h... yes 2024-04-25T21:02:59.696 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/ioctl.h usability... yes 2024-04-25T21:02:59.713 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/ioctl.h presence... yes 2024-04-25T21:02:59.713 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/ioctl.h... yes 2024-04-25T21:02:59.768 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/wait.h usability... yes 2024-04-25T21:02:59.786 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/wait.h presence... yes 2024-04-25T21:02:59.786 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/wait.h... yes 2024-04-25T21:02:59.806 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/types.h... (cached) yes 2024-04-25T21:02:59.810 INFO:tasks.workunit.client.0.smithi047.stdout:checking for strings.h... (cached) yes 2024-04-25T21:02:59.850 INFO:tasks.workunit.client.0.smithi047.stdout:checking err.h usability... yes 2024-04-25T21:02:59.866 INFO:tasks.workunit.client.0.smithi047.stdout:checking err.h presence... yes 2024-04-25T21:02:59.866 INFO:tasks.workunit.client.0.smithi047.stdout:checking for err.h... yes 2024-04-25T21:02:59.917 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/falloc.h usability... yes 2024-04-25T21:02:59.937 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/falloc.h presence... yes 2024-04-25T21:02:59.937 INFO:tasks.workunit.client.0.smithi047.stdout:checking for linux/falloc.h... yes 2024-04-25T21:02:59.991 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-25T21:03:00.021 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-25T21:03:00.021 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-25T21:03:00.069 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-25T21:03:00.113 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-25T21:03:00.113 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-25T21:03:00.160 INFO:tasks.workunit.client.0.smithi047.stdout:checking xfs/platform_defs.h usability... no 2024-04-25T21:03:00.195 INFO:tasks.workunit.client.0.smithi047.stdout:checking xfs/platform_defs.h presence... no 2024-04-25T21:03:00.195 INFO:tasks.workunit.client.0.smithi047.stdout:checking for xfs/platform_defs.h... no 2024-04-25T21:03:00.242 INFO:tasks.workunit.client.0.smithi047.stdout:checking btrfs/ioctl.h usability... no 2024-04-25T21:03:00.276 INFO:tasks.workunit.client.0.smithi047.stdout:checking btrfs/ioctl.h presence... no 2024-04-25T21:03:00.276 INFO:tasks.workunit.client.0.smithi047.stdout:checking for btrfs/ioctl.h... no 2024-04-25T21:03:00.319 INFO:tasks.workunit.client.0.smithi047.stdout:checking cifs/ioctl.h usability... no 2024-04-25T21:03:00.351 INFO:tasks.workunit.client.0.smithi047.stdout:checking cifs/ioctl.h presence... no 2024-04-25T21:03:00.351 INFO:tasks.workunit.client.0.smithi047.stdout:checking for cifs/ioctl.h... no 2024-04-25T21:03:00.402 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/mman.h usability... yes 2024-04-25T21:03:00.423 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/mman.h presence... yes 2024-04-25T21:03:00.424 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/mman.h... yes 2024-04-25T21:03:00.481 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/ext4.h usability... no 2024-04-25T21:03:00.516 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/ext4.h presence... no 2024-04-25T21:03:00.516 INFO:tasks.workunit.client.0.smithi047.stdout:checking for linux/ext4.h... no 2024-04-25T21:03:00.549 INFO:tasks.workunit.client.0.smithi047.stdout:checking for xfs/xfs_log_format.h... no 2024-04-25T21:03:00.584 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:00 smithi092 ceph-mon[32524]: pgmap v93: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-04-25T21:03:00.609 INFO:tasks.workunit.client.0.smithi047.stdout:checking for uuid_compare... no 2024-04-25T21:03:00.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:00 smithi047 ceph-mon[29195]: pgmap v93: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-04-25T21:03:00.652 INFO:tasks.workunit.client.0.smithi047.stdout:checking for uuid_compare in -luuid... yes 2024-04-25T21:03:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:00 smithi186 ceph-mon[32529]: pgmap v93: 97 pgs: 97 active+clean; 38 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-04-25T21:03:00.722 INFO:tasks.workunit.client.0.smithi047.stdout:checking for xfs/xfs.h... yes 2024-04-25T21:03:00.763 INFO:tasks.workunit.client.0.smithi047.stdout:checking for xfs/libxfs.h... no 2024-04-25T21:03:00.798 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-25T21:03:00.841 INFO:tasks.workunit.client.0.smithi047.stdout:checking for xfs/xqm.h... yes 2024-04-25T21:03:00.897 INFO:tasks.workunit.client.0.smithi047.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-25T21:03:00.942 INFO:tasks.workunit.client.0.smithi047.stdout:checking xfs/handle.h usability... yes 2024-04-25T21:03:00.963 INFO:tasks.workunit.client.0.smithi047.stdout:checking xfs/handle.h presence... yes 2024-04-25T21:03:00.963 INFO:tasks.workunit.client.0.smithi047.stdout:checking for xfs/handle.h... yes 2024-04-25T21:03:01.007 INFO:tasks.workunit.client.0.smithi047.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-25T21:03:01.054 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/acl.h usability... yes 2024-04-25T21:03:01.077 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/acl.h presence... yes 2024-04-25T21:03:01.077 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/acl.h... yes 2024-04-25T21:03:01.122 INFO:tasks.workunit.client.0.smithi047.stdout:checking acl/libacl.h usability... yes 2024-04-25T21:03:01.144 INFO:tasks.workunit.client.0.smithi047.stdout:checking acl/libacl.h presence... yes 2024-04-25T21:03:01.144 INFO:tasks.workunit.client.0.smithi047.stdout:checking for acl/libacl.h... yes 2024-04-25T21:03:01.190 INFO:tasks.workunit.client.0.smithi047.stdout:checking for acl_init in -lacl... yes 2024-04-25T21:03:01.266 INFO:tasks.workunit.client.0.smithi047.stdout:checking gdbm-ndbm.h usability... no 2024-04-25T21:03:01.302 INFO:tasks.workunit.client.0.smithi047.stdout:checking gdbm-ndbm.h presence... no 2024-04-25T21:03:01.302 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gdbm-ndbm.h... no 2024-04-25T21:03:01.344 INFO:tasks.workunit.client.0.smithi047.stdout:checking gdbm/ndbm.h usability... no 2024-04-25T21:03:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:checking gdbm/ndbm.h presence... no 2024-04-25T21:03:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:checking for gdbm/ndbm.h... no 2024-04-25T21:03:01.422 INFO:tasks.workunit.client.0.smithi047.stdout:checking ndbm.h usability... no 2024-04-25T21:03:01.457 INFO:tasks.workunit.client.0.smithi047.stdout:checking ndbm.h presence... no 2024-04-25T21:03:01.457 INFO:tasks.workunit.client.0.smithi047.stdout:checking for ndbm.h... no 2024-04-25T21:03:01.495 INFO:tasks.workunit.client.0.smithi047.stdout:checking libaio.h usability... yes 2024-04-25T21:03:01.517 INFO:tasks.workunit.client.0.smithi047.stdout:checking libaio.h presence... yes 2024-04-25T21:03:01.518 INFO:tasks.workunit.client.0.smithi047.stdout:checking for libaio.h... yes 2024-04-25T21:03:01.566 INFO:tasks.workunit.client.0.smithi047.stdout:checking liburing.h usability... no 2024-04-25T21:03:01.595 INFO:tasks.workunit.client.0.smithi047.stdout:checking liburing.h presence... no 2024-04-25T21:03:01.595 INFO:tasks.workunit.client.0.smithi047.stdout:checking for liburing.h... no 2024-04-25T21:03:01.637 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/capability.h usability... no 2024-04-25T21:03:01.671 INFO:tasks.workunit.client.0.smithi047.stdout:checking sys/capability.h presence... no 2024-04-25T21:03:01.671 INFO:tasks.workunit.client.0.smithi047.stdout:checking for sys/capability.h... no 2024-04-25T21:03:01.703 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/fiemap.h usability... yes 2024-04-25T21:03:01.718 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/fiemap.h presence... yes 2024-04-25T21:03:01.718 INFO:tasks.workunit.client.0.smithi047.stdout:checking for linux/fiemap.h... yes 2024-04-25T21:03:01.756 INFO:tasks.workunit.client.0.smithi047.stdout:checking for fallocate... yes 2024-04-25T21:03:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 -- 172.21.15.47:0/1654299359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa560106170 msgr2=0x7fa56010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 --2- 172.21.15.47:0/1654299359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa560106170 0x7fa56010dd40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa558009900 tx=0x7fa55802a410 comp rx=0 tx=0).stop 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 -- 172.21.15.47:0/1654299359 shutdown_connections 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 --2- 172.21.15.47:0/1654299359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa560106170 0x7fa56010dd40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 --2- 172.21.15.47:0/1654299359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa560101aa0 0x7fa560105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 --2- 172.21.15.47:0/1654299359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5601010f0 0x7fa5601014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 -- 172.21.15.47:0/1654299359 >> 172.21.15.47:0/1654299359 conn(0x7fa560077fd0 msgr2=0x7fa5600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 -- 172.21.15.47:0/1654299359 shutdown_connections 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.780+0000 7fa56767c640 1 -- 172.21.15.47:0/1654299359 wait complete. 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa56767c640 1 Processor -- start 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa56767c640 1 -- start start 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa56767c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5601010f0 0x7fa560198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa56767c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa560106170 0x7fa560198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa56767c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa56019cfc0 0x7fa56019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa56767c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601110e0 con 0x7fa5601010f0 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa56767c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa560110f60 con 0x7fa56019cfc0 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa56767c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa560111260 con 0x7fa560106170 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565418640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5601010f0 0x7fa560198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565418640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5601010f0 0x7fa560198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40266/0 (socket says 172.21.15.47:40266) 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565418640 1 -- 172.21.15.47:0/2470785040 learned_addr learned my addr 172.21.15.47:0/2470785040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa564c17640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa560106170 0x7fa560198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565c19640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa56019cfc0 0x7fa56019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565418640 1 -- 172.21.15.47:0/2470785040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa560106170 msgr2=0x7fa560198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565418640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa560106170 0x7fa560198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565418640 1 -- 172.21.15.47:0/2470785040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa56019cfc0 msgr2=0x7fa56019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565418640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa56019cfc0 0x7fa56019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.781+0000 7fa565418640 1 -- 172.21.15.47:0/2470785040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56019dab0 con 0x7fa5601010f0 2024-04-25T21:03:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.782+0000 7fa565418640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5601010f0 0x7fa560198700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00ea80 tx=0x7fa55c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.782+0000 7fa5567fc640 1 -- 172.21.15.47:0/2470785040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00ce40 con 0x7fa5601010f0 2024-04-25T21:03:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.782+0000 7fa56767c640 1 -- 172.21.15.47:0/2470785040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56019dda0 con 0x7fa5601010f0 2024-04-25T21:03:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.782+0000 7fa5567fc640 1 -- 172.21.15.47:0/2470785040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa55c005020 con 0x7fa5601010f0 2024-04-25T21:03:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.782+0000 7fa5567fc640 1 -- 172.21.15.47:0/2470785040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0116d0 con 0x7fa5601010f0 2024-04-25T21:03:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.782+0000 7fa56767c640 1 -- 172.21.15.47:0/2470785040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5601a5590 con 0x7fa5601010f0 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.784+0000 7fa5567fc640 1 -- 172.21.15.47:0/2470785040 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa55c0054d0 con 0x7fa5601010f0 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.784+0000 7fa56a392740 1 Processor -- start 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.784+0000 7fa56a392740 1 -- start start 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa56a392740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f0158fba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa56a392740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0159062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa56a392740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015908bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa56a392740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f015860970 con 0x55f0158fbf40 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa56a392740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f0157fd4a0 con 0x55f0158fb620 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa56a392740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f01585c930 con 0x55f015906800 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa5567fc640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa54c0a3dc0 0x7fa54c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565418640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0159062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565c19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f0158fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565c19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f0158fba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:53166/0 (socket says 172.21.15.47:53166) 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565c19640 1 -- 172.21.15.47:0/3120164035 learned_addr learned my addr 172.21.15.47:0/3120164035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565418640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0159062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40274/0 (socket says 172.21.15.47:40274) 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565c19640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa54c0a3dc0 0x7fa54c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa564c17640 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015908bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa5567fc640 1 -- 172.21.15.47:0/2470785040 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fa55c0d0a20 con 0x7fa5601010f0 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565418640 1 -- 172.21.15.47:0/3120164035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 msgr2=0x55f015908bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565418640 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015908bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565418640 1 -- 172.21.15.47:0/3120164035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 msgr2=0x55f0158fba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565418640 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f0158fba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565418640 1 -- 172.21.15.47:0/3120164035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f015573210 con 0x55f0158fbf40 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa564c17640 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015908bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.785+0000 7fa565c19640 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f0158fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa565418640 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0159062c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0ce180 tx=0x7fa55c098e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6085056bfb148f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa565c19640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa54c0a3dc0 0x7fa54c0a6280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5580096c0 tx=0x7fa5580058d0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa54b7fe640 1 -- 172.21.15.47:0/3120164035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c01f070 con 0x55f0158fbf40 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa54b7fe640 1 -- 172.21.15.47:0/3120164035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa55c079e50 con 0x55f0158fbf40 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa54b7fe640 1 -- 172.21.15.47:0/3120164035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0052f0 con 0x55f0158fbf40 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa56a392740 1 -- 172.21.15.47:0/3120164035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 msgr2=0x55f0159062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa56a392740 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0159062c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0ce180 tx=0x7fa55c098e50 comp rx=0 tx=0).stop 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa56a392740 1 -- 172.21.15.47:0/3120164035 shutdown_connections 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.786+0000 7fa56a392740 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015908bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0159062c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 --2- 172.21.15.47:0/3120164035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f0158fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 -- 172.21.15.47:0/3120164035 >> 172.21.15.47:0/3120164035 conn(0x55f0158f7b30 msgr2=0x55f0158f7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 -- 172.21.15.47:0/3120164035 shutdown_connections 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 -- 172.21.15.47:0/3120164035 wait complete. 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 -- start start 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 Processor -- start 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015906240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f015903530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015903c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f0159093d0 con 0x55f0158fbf40 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f0157fb0b0 con 0x55f0158fb620 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.787+0000 7fa56a392740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f015909550 con 0x55f015906800 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa56a392740 10 client.0 _ll_register_callbacks cb 0x55f0157e8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa564c17640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f015903530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565418640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015906240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa564c17640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f015903530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40290/0 (socket says 172.21.15.47:40290) 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565418640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015906240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:53178/0 (socket says 172.21.15.47:53178) 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa564c17640 1 -- 172.21.15.47:0/835490154 learned_addr learned my addr 172.21.15.47:0/835490154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565c19640 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015903c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565418640 1 -- 172.21.15.47:0/835490154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 msgr2=0x55f015903c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565418640 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015903c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565418640 1 -- 172.21.15.47:0/835490154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 msgr2=0x55f015903530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565418640 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f015903530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565418640 1 -- 172.21.15.47:0/835490154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01563f620 con 0x55f0158fb620 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565c19640 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015903c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa564c17640 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f015903530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa565418640 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015906240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00efd0 tx=0x7fa55c09cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c2e55d91b53de67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:03:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa548ff9640 1 -- 172.21.15.47:0/835490154 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c01f050 con 0x55f0158fb620 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa548ff9640 1 -- 172.21.15.47:0/835490154 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fa55c0a51b0 con 0x55f0158fb620 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.788+0000 7fa56a392740 1 -- 172.21.15.47:0/835490154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa55c079e50 con 0x55f0158fb620 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa548ff9640 1 -- 172.21.15.47:0/835490154 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa55c0a5430 con 0x55f0158fb620 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa56a392740 10 client.25283 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa56a392740 1 -- 172.21.15.47:0/835490154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f0159093d0 con 0x55f0158fb620 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa548ff9640 1 -- 172.21.15.47:0/835490154 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa55c079e50 con 0x55f0158fb620 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa548ff9640 1 -- 172.21.15.47:0/835490154 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa55c00f040 con 0x55f0158fb620 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa56a392740 10 client.25283 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa56a392740 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa56a392740 20 client.25283 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa56a392740 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f015916540 0x55f015918930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa56a392740 4 client.25283 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa56a392740 1 -- 172.21.15.47:0/835490154 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f01586e390 con 0x55f015916540 2024-04-25T21:03:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.789+0000 7fa564c17640 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f015916540 0x55f015918930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa564c17640 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f015916540 0x55f015918930 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af02fd1b1e6a5fd server_cookie=9ccac91517868d17 in_seq=0 out_seq=0 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa548ff9640 10 client.25283 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa548ff9640 1 -- 172.21.15.47:0/835490154 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f01586e390 con 0x55f015916540 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa548ff9640 10 client.25283 handle_command_reply: tid=0 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 client.25283 shutdown 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 -- 172.21.15.47:0/835490154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 msgr2=0x55f015906240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015906240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00efd0 tx=0x7fa55c09cc00 comp rx=0 tx=0).stop 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 -- 172.21.15.47:0/835490154 shutdown_connections 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f015916540 0x55f015918930 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015903c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f015903530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa548ff9640 0 client.25283 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 --2- 172.21.15.47:0/835490154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015906240 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.790+0000 7fa56a392740 1 -- 172.21.15.47:0/835490154 >> 172.21.15.47:0/835490154 conn(0x55f0158f7b30 msgr2=0x55f0158fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.791+0000 7fa56a392740 1 -- 172.21.15.47:0/835490154 shutdown_connections 2024-04-25T21:03:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.791+0000 7fa56a392740 1 -- 172.21.15.47:0/835490154 wait complete. 2024-04-25T21:03:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.791+0000 7fa56a392740 20 client.25283 trim_cache size 0 max 16384 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.806+0000 7fa56a392740 1 Processor -- start 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.806+0000 7fa56a392740 1 -- start start 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.806+0000 7fa56a392740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015912290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa56a392740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0158fe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa56a392740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015905d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa56a392740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601b0450 con 0x55f0158fbf40 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa56a392740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601adcc0 con 0x55f0158fb620 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa56a392740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601abb30 con 0x55f015906800 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0158fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa565418640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015912290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0158fe970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40298/0 (socket says 172.21.15.47:40298) 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 -- 172.21.15.47:0/533311697 learned_addr learned my addr 172.21.15.47:0/533311697 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa565c19640 1 --2- 172.21.15.47:0/533311697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015905d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 -- 172.21.15.47:0/533311697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 msgr2=0x55f015905d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 --2- 172.21.15.47:0/533311697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015905d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 -- 172.21.15.47:0/533311697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 msgr2=0x55f015912290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 --2- 172.21.15.47:0/533311697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015912290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 -- 172.21.15.47:0/533311697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0158f3500 con 0x55f0158fbf40 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa564c17640 1 --2- 172.21.15.47:0/533311697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0158fe970 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55f015918fc0 tx=0x7fa550005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c93ed48b530ba22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.807+0000 7fa54affd640 1 -- 172.21.15.47:0/533311697 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5500120d0 con 0x55f0158fbf40 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa54affd640 1 -- 172.21.15.47:0/533311697 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa550012270 con 0x55f0158fbf40 2024-04-25T21:03:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa54affd640 1 -- 172.21.15.47:0/533311697 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004dc0 con 0x55f0158fbf40 2024-04-25T21:03:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 -- 172.21.15.47:0/533311697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 msgr2=0x55f0158fe970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 --2- 172.21.15.47:0/533311697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0158fe970 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55f015918fc0 tx=0x7fa550005f90 comp rx=0 tx=0).stop 2024-04-25T21:03:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 -- 172.21.15.47:0/533311697 shutdown_connections 2024-04-25T21:03:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 --2- 172.21.15.47:0/533311697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f015906800 0x55f015905d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 --2- 172.21.15.47:0/533311697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0158fbf40 0x55f0158fe970 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 --2- 172.21.15.47:0/533311697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f015912290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 -- 172.21.15.47:0/533311697 >> 172.21.15.47:0/533311697 conn(0x55f0158f7b30 msgr2=0x55f0158fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 -- 172.21.15.47:0/533311697 shutdown_connections 2024-04-25T21:03:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.808+0000 7fa56a392740 1 -- 172.21.15.47:0/533311697 wait complete. 2024-04-25T21:03:01.811 INFO:tasks.workunit.client.0.smithi047.stdout:checking for open_by_handle_at... yes 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 1 -- start start 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 1 Processor -- start 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f01590e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0158fbf40 0x55f015909d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f015906800 0x55f01590a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601ac180 con 0x55f015906800 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601ad5f0 con 0x55f0158fb620 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601ac850 con 0x55f0158fbf40 2024-04-25T21:03:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa56a392740 10 client.0 _ll_register_callbacks cb 0x7fa55c005ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa565418640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f01590e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa565c19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f015906800 0x55f01590a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa565418640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f01590e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:53196/0 (socket says 172.21.15.47:53196) 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa565418640 1 -- 172.21.15.47:0/2332192406 learned_addr learned my addr 172.21.15.47:0/2332192406 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.809+0000 7fa564c17640 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0158fbf40 0x55f015909d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa565418640 1 -- 172.21.15.47:0/2332192406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0158fbf40 msgr2=0x55f015909d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa565418640 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0158fbf40 0x55f015909d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa565418640 1 -- 172.21.15.47:0/2332192406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f015906800 msgr2=0x55f01590a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa565418640 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f015906800 0x55f01590a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa565418640 1 -- 172.21.15.47:0/2332192406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01590a800 con 0x55f0158fb620 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa565c19640 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f015906800 0x55f01590a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa565418640 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f01590e820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa55c087110 tx=0x7fa55c09af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88d8124a9fd3c0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:03:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa548ff9640 1 -- 172.21.15.47:0/2332192406 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0bbb90 con 0x55f0158fb620 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa56a392740 1 -- 172.21.15.47:0/2332192406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa550012270 con 0x55f0158fb620 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa548ff9640 1 -- 172.21.15.47:0/2332192406 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fa55c0bbd30 con 0x55f0158fb620 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa548ff9640 1 -- 172.21.15.47:0/2332192406 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa55c087140 con 0x55f0158fb620 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa56a392740 10 client.25289 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa56a392740 1 -- 172.21.15.47:0/2332192406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f01590a800 con 0x55f0158fb620 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.810+0000 7fa548ff9640 1 -- 172.21.15.47:0/2332192406 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa550012270 con 0x55f0158fb620 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa548ff9640 1 -- 172.21.15.47:0/2332192406 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa55c09b9b0 con 0x55f0158fb620 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa56a392740 10 client.25289 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa56a392740 10 client.25289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa56a392740 20 client.25289 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa56a392740 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f01597c940 0x55f01597ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa56a392740 4 client.25289 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa56a392740 1 -- 172.21.15.47:0/2332192406 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f015868200 con 0x55f01597c940 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa564c17640 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f01597c940 0x55f01597ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa564c17640 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f01597c940 0x55f01597ed60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4438e63656417a2f server_cookie=3e5956a8bd2c4b1b in_seq=0 out_seq=0 2024-04-25T21:03:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.811+0000 7fa548ff9640 10 client.25289 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa548ff9640 1 -- 172.21.15.47:0/2332192406 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f015868200 con 0x55f01597c940 2024-04-25T21:03:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa548ff9640 10 client.25289 handle_command_reply: tid=0 2024-04-25T21:03:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 client.25289 shutdown 2024-04-25T21:03:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 -- 172.21.15.47:0/2332192406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 msgr2=0x55f01590e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f01590e820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa55c087110 tx=0x7fa55c09af90 comp rx=0 tx=0).stop 2024-04-25T21:03:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 -- 172.21.15.47:0/2332192406 shutdown_connections 2024-04-25T21:03:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55f01597c940 0x55f01597ed60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f015906800 0x55f01590a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa548ff9640 0 client.25289 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0158fbf40 0x55f015909d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 --2- 172.21.15.47:0/2332192406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f0158fb620 0x55f01590e820 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.820+0000 7fa56a392740 1 -- 172.21.15.47:0/2332192406 >> 172.21.15.47:0/2332192406 conn(0x55f0158f7b30 msgr2=0x55f0158f6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56a392740 1 -- 172.21.15.47:0/2332192406 shutdown_connections 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56a392740 1 -- 172.21.15.47:0/2332192406 wait complete. 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56a392740 20 client.25289 trim_cache size 0 max 16384 2024-04-25T21:03:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:03:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:03:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:03:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:03:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 -- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa54c0a3dc0 msgr2=0x7fa54c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa54c0a3dc0 0x7fa54c0a6280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5580096c0 tx=0x7fa5580058d0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 -- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5601010f0 msgr2=0x7fa560198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5601010f0 0x7fa560198700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00ea80 tx=0x7fa55c00ef40 comp rx=0 tx=0).stop 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 -- 172.21.15.47:0/2470785040 shutdown_connections 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa54c0a3dc0 0x7fa54c0a6280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa56019cfc0 0x7fa56019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa560106170 0x7fa560198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 --2- 172.21.15.47:0/2470785040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5601010f0 0x7fa560198700 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 -- 172.21.15.47:0/2470785040 >> 172.21.15.47:0/2470785040 conn(0x7fa560077fd0 msgr2=0x7fa560069990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.821+0000 7fa56767c640 1 -- 172.21.15.47:0/2470785040 shutdown_connections 2024-04-25T21:03:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:01.822+0000 7fa56767c640 1 -- 172.21.15.47:0/2470785040 wait complete. 2024-04-25T21:03:01.850 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/fs.h usability... yes 2024-04-25T21:03:01.866 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/fs.h presence... yes 2024-04-25T21:03:01.867 INFO:tasks.workunit.client.0.smithi047.stdout:checking for linux/fs.h... yes 2024-04-25T21:03:01.926 INFO:tasks.workunit.client.0.smithi047.stdout:checking btrfsutil.h usability... no 2024-04-25T21:03:01.955 INFO:tasks.workunit.client.0.smithi047.stdout:checking btrfsutil.h presence... no 2024-04-25T21:03:01.955 INFO:tasks.workunit.client.0.smithi047.stdout:checking for btrfsutil.h... no 2024-04-25T21:03:01.996 INFO:tasks.workunit.client.0.smithi047.stdout:checking for copy_file_range... yes 2024-04-25T21:03:02.045 INFO:tasks.workunit.client.0.smithi047.stdout:checking for SEEK_DATA... yes 2024-04-25T21:03:02.103 INFO:tasks.workunit.client.0.smithi047.stdout:checking for BMV_OF_SHARED... yes 2024-04-25T21:03:02.149 INFO:tasks.workunit.client.0.smithi047.stdout:checking for nftw... yes 2024-04-25T21:03:02.196 INFO:tasks.workunit.client.0.smithi047.stdout:checking for RLIMIT_NOFILE... yes 2024-04-25T21:03:02.240 INFO:tasks.workunit.client.0.smithi047.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-25T21:03:02.305 INFO:tasks.workunit.client.0.smithi047.stdout:checking for FICLONE... yes 2024-04-25T21:03:02.352 INFO:tasks.workunit.client.0.smithi047.stdout:checking for renameat2... yes 2024-04-25T21:03:02.395 INFO:tasks.workunit.client.0.smithi047.stdout:checking for reallocarray... yes 2024-04-25T21:03:02.460 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct mount_attr... yes 2024-04-25T21:03:02.522 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-25T21:03:02.582 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-25T21:03:02.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:02 smithi047 ceph-mon[29195]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-25T21:03:02.637 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-25T21:03:02.695 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-25T21:03:02.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:02 smithi186 ceph-mon[32529]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-25T21:03:02.753 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-25T21:03:02.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:02 smithi092 ceph-mon[32524]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-25T21:03:02.812 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-25T21:03:02.876 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-25T21:03:02.916 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/btrfs.h usability... yes 2024-04-25T21:03:02.933 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/btrfs.h presence... yes 2024-04-25T21:03:02.933 INFO:tasks.workunit.client.0.smithi047.stdout:checking for linux/btrfs.h... yes 2024-04-25T21:03:02.981 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-25T21:03:02.997 INFO:tasks.workunit.client.0.smithi047.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-25T21:03:02.997 INFO:tasks.workunit.client.0.smithi047.stdout:checking for linux/btrfs_tree.h... yes 2024-04-25T21:03:03.035 INFO:tasks.workunit.client.0.smithi047.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-25T21:03:03.066 INFO:tasks.workunit.client.0.smithi047.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-25T21:03:03.086 INFO:tasks.workunit.client.0.smithi047.stdout:configure: creating ./config.status 2024-04-25T21:03:04.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:04 smithi047 ceph-mon[29195]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 585 KiB/s wr, 32 op/s 2024-04-25T21:03:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:04 smithi186 ceph-mon[32529]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 585 KiB/s wr, 32 op/s 2024-04-25T21:03:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:04 smithi092 ceph-mon[32524]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 585 KiB/s wr, 32 op/s 2024-04-25T21:03:05.221 INFO:tasks.workunit.client.0.smithi047.stdout:config.status: creating include/builddefs 2024-04-25T21:03:05.239 INFO:tasks.workunit.client.0.smithi047.stdout:config.status: creating include/config.h 2024-04-25T21:03:05.253 INFO:tasks.workunit.client.0.smithi047.stdout:config.status: executing libtool commands 2024-04-25T21:03:05.529 INFO:tasks.workunit.client.0.smithi047.stdout:Building include 2024-04-25T21:03:05.529 INFO:tasks.workunit.client.0.smithi047.stdout:Building lib 2024-04-25T21:03:05.529 INFO:tasks.workunit.client.0.smithi047.stdout:Building tests 2024-04-25T21:03:05.536 INFO:tasks.workunit.client.0.smithi047.stdout: [LTDEP] 2024-04-25T21:03:05.536 INFO:tasks.workunit.client.0.smithi047.stdout:Building btrfs 2024-04-25T21:03:05.537 INFO:tasks.workunit.client.0.smithi047.stdout:Building ceph 2024-04-25T21:03:05.537 INFO:tasks.workunit.client.0.smithi047.stdout:Building cifs 2024-04-25T21:03:05.549 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-25T21:03:05.549 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-25T21:03:05.577 INFO:tasks.workunit.client.0.smithi047.stdout:Building ext4 2024-04-25T21:03:05.579 INFO:tasks.workunit.client.0.smithi047.stdout:Building f2fs 2024-04-25T21:03:05.585 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-25T21:03:05.586 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-25T21:03:05.600 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-25T21:03:05.610 INFO:tasks.workunit.client.0.smithi047.stdout:Building generic 2024-04-25T21:03:05.644 INFO:tasks.workunit.client.0.smithi047.stdout:Building nfs 2024-04-25T21:03:05.653 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-25T21:03:05.660 INFO:tasks.workunit.client.0.smithi047.stdout:Building ocfs2 2024-04-25T21:03:05.668 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-25T21:03:05.670 INFO:tasks.workunit.client.0.smithi047.stdout:Building overlay 2024-04-25T21:03:05.685 INFO:tasks.workunit.client.0.smithi047.stdout:Building perf 2024-04-25T21:03:05.692 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-25T21:03:05.708 INFO:tasks.workunit.client.0.smithi047.stdout:Building selftest 2024-04-25T21:03:05.710 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-25T21:03:05.715 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-25T21:03:05.732 INFO:tasks.workunit.client.0.smithi047.stdout:Building shared 2024-04-25T21:03:05.740 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-25T21:03:05.758 INFO:tasks.workunit.client.0.smithi047.stdout:Building tmpfs 2024-04-25T21:03:05.760 INFO:tasks.workunit.client.0.smithi047.stdout:Building udf 2024-04-25T21:03:05.764 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-25T21:03:05.768 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-25T21:03:05.783 INFO:tasks.workunit.client.0.smithi047.stdout:Building xfs 2024-04-25T21:03:05.804 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] dataascii.lo 2024-04-25T21:03:05.804 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] databin.lo 2024-04-25T21:03:05.827 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-25T21:03:05.938 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] datapid.lo 2024-04-25T21:03:05.945 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] file_lock.lo 2024-04-25T21:03:05.964 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] forker.lo 2024-04-25T21:03:05.970 INFO:tasks.workunit.client.0.smithi047.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-25T21:03:06.093 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] pattern.lo 2024-04-25T21:03:06.131 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] open_flags.lo 2024-04-25T21:03:06.153 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] random_range.lo 2024-04-25T21:03:06.167 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] string_to_tokens.lo 2024-04-25T21:03:06.226 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] str_to_bytes.lo 2024-04-25T21:03:06.284 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] tlibio.lo 2024-04-25T21:03:06.290 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] write_log.lo 2024-04-25T21:03:06.387 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] random.lo 2024-04-25T21:03:06.538 INFO:tasks.workunit.client.0.smithi047.stdout: [LD] 2024-04-25T21:03:06.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:06 smithi047 ceph-mon[29195]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-04-25T21:03:06.668 INFO:tasks.workunit.client.0.smithi047.stdout:Building ltp 2024-04-25T21:03:06.668 INFO:tasks.workunit.client.0.smithi047.stdout:Building src 2024-04-25T21:03:06.668 INFO:tasks.workunit.client.0.smithi047.stdout:Building m4 2024-04-25T21:03:06.668 INFO:tasks.workunit.client.0.smithi047.stdout:Building common 2024-04-25T21:03:06.675 INFO:tasks.workunit.client.0.smithi047.stdout: [DEP] 2024-04-25T21:03:06.691 INFO:tasks.workunit.client.0.smithi047.stdout: [DEP] 2024-04-25T21:03:06.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:06 smithi186 ceph-mon[32529]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-04-25T21:03:06.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:06 smithi092 ceph-mon[32524]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-04-25T21:03:06.880 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] doio 2024-04-25T21:03:06.880 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fsstress 2024-04-25T21:03:06.880 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fsx 2024-04-25T21:03:07.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:07 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:07.636 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] growfiles 2024-04-25T21:03:07.676 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] iogen 2024-04-25T21:03:07.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:07 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:07.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:07 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:08.165 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-stress 2024-04-25T21:03:08.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:08 smithi047 ceph-mon[29195]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-04-25T21:03:08.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:08 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:08.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:08 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:08.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:08 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:03:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:08 smithi186 ceph-mon[32529]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-04-25T21:03:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:08 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:08 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:08 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:03:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:08 smithi092 ceph-mon[32524]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-04-25T21:03:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:08 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:08 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:08 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:03:08.964 INFO:tasks.workunit.client.0.smithi047.stdout: [DEP] 2024-04-25T21:03:08.965 INFO:tasks.workunit.client.0.smithi047.stdout: [DEP] 2024-04-25T21:03:08.965 INFO:tasks.workunit.client.0.smithi047.stdout: [DEP] 2024-04-25T21:03:08.968 INFO:tasks.workunit.client.0.smithi047.stdout: [DEP] 2024-04-25T21:03:09.202 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] dirstress 2024-04-25T21:03:09.202 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fill 2024-04-25T21:03:09.202 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fill2 2024-04-25T21:03:09.204 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] getpagesize 2024-04-25T21:03:09.354 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] holes 2024-04-25T21:03:09.381 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] lstat64 2024-04-25T21:03:09.443 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] nametest 2024-04-25T21:03:09.464 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] permname 2024-04-25T21:03:09.586 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] randholes 2024-04-25T21:03:09.594 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] runas 2024-04-25T21:03:09.701 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] truncfile 2024-04-25T21:03:09.703 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] usemem 2024-04-25T21:03:09.796 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] mmapcat 2024-04-25T21:03:09.873 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] append_reader 2024-04-25T21:03:09.898 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] append_writer 2024-04-25T21:03:09.914 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] dirperf 2024-04-25T21:03:09.959 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] metaperf 2024-04-25T21:03:10.040 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] devzero 2024-04-25T21:03:10.060 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] feature 2024-04-25T21:03:10.157 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] alloc 2024-04-25T21:03:10.221 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fault 2024-04-25T21:03:10.243 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fstest 2024-04-25T21:03:10.283 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_access_root 2024-04-25T21:03:10.383 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] godown 2024-04-25T21:03:10.393 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] resvtest 2024-04-25T21:03:10.445 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] writemod 2024-04-25T21:03:10.512 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] writev_on_pagefault 2024-04-25T21:03:10.565 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] makeextents 2024-04-25T21:03:10.589 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] itrash 2024-04-25T21:03:10.589 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] rename 2024-04-25T21:03:10.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:10 smithi047 ceph-mon[29195]: pgmap v98: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-04-25T21:03:10.682 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] multi_open_unlink 2024-04-25T21:03:10.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:10 smithi186 ceph-mon[32529]: pgmap v98: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-04-25T21:03:10.739 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] unwritten_sync 2024-04-25T21:03:10.746 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] genhashnames 2024-04-25T21:03:10.754 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_holes 2024-04-25T21:03:10.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:10 smithi092 ceph-mon[32524]: pgmap v98: 97 pgs: 97 active+clean; 54 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-04-25T21:03:10.863 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_writev 2024-04-25T21:03:10.895 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_truncate_cmtime 2024-04-25T21:03:10.927 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] dirhash_collide 2024-04-25T21:03:10.935 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_rename_overwrite 2024-04-25T21:03:11.008 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] holetest 2024-04-25T21:03:11.057 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_truncate_self 2024-04-25T21:03:11.071 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] af_unix 2024-04-25T21:03:11.133 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_stale_pmd 2024-04-25T21:03:11.198 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_cow_race 2024-04-25T21:03:11.228 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_fallocate 2024-04-25T21:03:11.244 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fsync-err 2024-04-25T21:03:11.294 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_write_ro 2024-04-25T21:03:11.360 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-25T21:03:11.397 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-25T21:03:11.432 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_ofd_locks 2024-04-25T21:03:11.456 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_collision 2024-04-25T21:03:11.517 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] mmap-write-concurrent 2024-04-25T21:03:11.617 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_get_file_time 2024-04-25T21:03:11.667 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_create_short_dirs 2024-04-25T21:03:11.682 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_create_long_dirs 2024-04-25T21:03:11.700 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_enospc 2024-04-25T21:03:11.762 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_writev_overlap 2024-04-25T21:03:11.851 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] checkpoint_journal 2024-04-25T21:03:11.852 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] mmap-rw-fault 2024-04-25T21:03:11.902 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] allocstale 2024-04-25T21:03:11.944 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_cow_memory_failure 2024-04-25T21:03:12.004 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fake-dump-rootino 2024-04-25T21:03:12.044 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] dio-buf-fault 2024-04-25T21:03:12.074 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] rewinddir-test 2024-04-25T21:03:12.134 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] readdir-while-renames 2024-04-25T21:03:12.166 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] xfsctl 2024-04-25T21:03:12.190 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] bstat 2024-04-25T21:03:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mtab 2024-04-25T21:03:12.309 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] getdevicesize 2024-04-25T21:03:12.370 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] preallo_rw_pattern_reader 2024-04-25T21:03:12.401 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] preallo_rw_pattern_writer 2024-04-25T21:03:12.463 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] ftrunc 2024-04-25T21:03:12.491 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] trunc 2024-04-25T21:03:12.550 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fs_perms 2024-04-25T21:03:12.556 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] testx 2024-04-25T21:03:12.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:12 smithi047 ceph-mon[29195]: pgmap v99: 97 pgs: 97 active+clean; 57 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2024-04-25T21:03:12.622 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] looptest 2024-04-25T21:03:12.660 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] locktest 2024-04-25T21:03:12.684 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] unwritten_mmap 2024-04-25T21:03:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:12 smithi186 ceph-mon[32529]: pgmap v99: 97 pgs: 97 active+clean; 57 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2024-04-25T21:03:12.726 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] bulkstat_unlink_test 2024-04-25T21:03:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:12 smithi092 ceph-mon[32524]: pgmap v99: 97 pgs: 97 active+clean; 57 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2024-04-25T21:03:12.832 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] deduperace 2024-04-25T21:03:12.839 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] bulkstat_unlink_test_modified 2024-04-25T21:03:12.933 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_dir_offset 2024-04-25T21:03:12.983 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_futimens 2024-04-25T21:03:13.043 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_immutable 2024-04-25T21:03:13.061 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] stale_handle 2024-04-25T21:03:13.103 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] pwrite_mmap_blocked 2024-04-25T21:03:13.129 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_dir_offset2 2024-04-25T21:03:13.232 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] seek_sanity_test 2024-04-25T21:03:13.261 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] seek_copy_test 2024-04-25T21:03:13.326 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_readdir_1 2024-04-25T21:03:13.441 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_readdir_2 2024-04-25T21:03:13.471 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fsync-tester 2024-04-25T21:03:13.605 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] nsexec 2024-04-25T21:03:13.714 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] cloner 2024-04-25T21:03:13.759 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] renameat2 2024-04-25T21:03:13.772 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_getcwd 2024-04-25T21:03:13.827 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] e4compact 2024-04-25T21:03:13.908 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] test-nextquota 2024-04-25T21:03:13.949 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] punch-alternating 2024-04-25T21:03:13.955 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-25T21:03:14.039 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] listxattr 2024-04-25T21:03:14.076 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] dio-interleaved 2024-04-25T21:03:14.134 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_dir_type 2024-04-25T21:03:14.134 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] dio-invalidate-cache 2024-04-25T21:03:14.189 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] stat_test 2024-04-25T21:03:14.249 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_encrypted_d_revalidate 2024-04-25T21:03:14.297 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] attr_replace_test 2024-04-25T21:03:14.369 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] swapon 2024-04-25T21:03:14.433 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] mkswap 2024-04-25T21:03:14.482 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_attr_corruption 2024-04-25T21:03:14.531 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_open_tmpfiles 2024-04-25T21:03:14.533 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fscrypt-crypt-util 2024-04-25T21:03:14.584 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] bulkstat_null_ocount 2024-04-25T21:03:14.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:14 smithi047 ceph-mon[29195]: pgmap v100: 97 pgs: 97 active+clean; 57 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2024-04-25T21:03:14.633 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] splice-test 2024-04-25T21:03:14.704 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] chprojid_fail 2024-04-25T21:03:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:14 smithi186 ceph-mon[32529]: pgmap v100: 97 pgs: 97 active+clean; 57 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2024-04-25T21:03:14.744 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] detached_mounts_propagation 2024-04-25T21:03:14.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:14 smithi092 ceph-mon[32524]: pgmap v100: 97 pgs: 97 active+clean; 57 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2024-04-25T21:03:14.824 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] ext4_resize 2024-04-25T21:03:14.870 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_readdir_3 2024-04-25T21:03:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 -- 172.21.15.47:0/273404231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d4106f30 msgr2=0x7f74d4107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 --2- 172.21.15.47:0/273404231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d4106f30 0x7f74d4107310 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009930 tx=0x7f74c802a420 comp rx=0 tx=0).stop 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 -- 172.21.15.47:0/273404231 shutdown_connections 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 --2- 172.21.15.47:0/273404231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d4100e10 0x7f74d410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 --2- 172.21.15.47:0/273404231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d4100450 0x7f74d41008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 --2- 172.21.15.47:0/273404231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d4106f30 0x7f74d4107310 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 -- 172.21.15.47:0/273404231 >> 172.21.15.47:0/273404231 conn(0x7f74d40773e0 msgr2=0x7f74d40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 -- 172.21.15.47:0/273404231 shutdown_connections 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.939+0000 7f74daa7c640 1 -- 172.21.15.47:0/273404231 wait complete. 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.940+0000 7f74daa7c640 1 Processor -- start 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.940+0000 7f74daa7c640 1 -- start start 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.940+0000 7f74daa7c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d4100450 0x7f74d4073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.940+0000 7f74daa7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d4100e10 0x7f74d4073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.940+0000 7f74daa7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d4106f30 0x7f74d406c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.940+0000 7f74daa7c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4112270 con 0x7f74d4100e10 2024-04-25T21:03:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.940+0000 7f74daa7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41120f0 con 0x7f74d4106f30 2024-04-25T21:03:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.940+0000 7f74daa7c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41123f0 con 0x7f74d4100450 2024-04-25T21:03:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d4106f30 0x7f74d406c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d4106f30 0x7f74d406c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:47910/0 (socket says 172.21.15.47:47910) 2024-04-25T21:03:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 -- 172.21.15.47:0/2644349322 learned_addr learned my addr 172.21.15.47:0/2644349322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d8818640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d4100450 0x7f74d4073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 -- 172.21.15.47:0/2644349322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d4100450 msgr2=0x7f74d4073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d4100450 0x7f74d4073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 -- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d4100e10 msgr2=0x7f74d4073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d4100e10 0x7f74d4073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 -- 172.21.15.47:0/2644349322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d406d220 con 0x7f74d4106f30 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.941+0000 7f74d9019640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d4106f30 0x7f74d406c960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00eb10 tx=0x7f74cc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.942+0000 7f74d1ffb640 1 -- 172.21.15.47:0/2644349322 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc004b20 con 0x7f74d4106f30 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.942+0000 7f74d1ffb640 1 -- 172.21.15.47:0/2644349322 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74cc004cc0 con 0x7f74d4106f30 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.942+0000 7f74daa7c640 1 -- 172.21.15.47:0/2644349322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d406d510 con 0x7f74d4106f30 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.942+0000 7f74d1ffb640 1 -- 172.21.15.47:0/2644349322 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc010670 con 0x7f74d4106f30 2024-04-25T21:03:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.942+0000 7f74daa7c640 1 -- 172.21.15.47:0/2644349322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d4071510 con 0x7f74d4106f30 2024-04-25T21:03:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74dd790740 1 Processor -- start 2024-04-25T21:03:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74d1ffb640 1 -- 172.21.15.47:0/2644349322 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f74cc005000 con 0x7f74d4106f30 2024-04-25T21:03:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74dd790740 1 -- start start 2024-04-25T21:03:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74dd790740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9eda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74dd790740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74dd790740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74dd790740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c24c952970 con 0x55c24c9edf40 2024-04-25T21:03:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74dd790740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c24c8ef4a0 con 0x55c24c9ed620 2024-04-25T21:03:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.946+0000 7f74dd790740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c24c94e930 con 0x55c24c9f8800 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d8818640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d8818640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57562/0 (socket says 172.21.15.47:57562) 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d8818640 1 -- 172.21.15.47:0/95158149 learned_addr learned my addr 172.21.15.47:0/95158149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d3fff640 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d3fff640 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d1ffb640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f74c40a3e40 0x7f74c40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d1ffb640 1 -- 172.21.15.47:0/2644349322 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f74cc0d0d90 con 0x7f74d4106f30 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d9019640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f74c40a3e40 0x7f74c40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d8818640 1 -- 172.21.15.47:0/95158149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 msgr2=0x55c24c9fabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d8818640 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9fabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d8818640 1 -- 172.21.15.47:0/95158149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 msgr2=0x55c24c9eda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d8818640 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9eda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.947+0000 7f74d8818640 1 -- 172.21.15.47:0/95158149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c24c665210 con 0x55c24c9edf40 2024-04-25T21:03:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.948+0000 7f74d8818640 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f82c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f74c802a940 tx=0x7f74c8031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aecc81df2d98ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.948+0000 7f74d9019640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f74c40a3e40 0x7f74c40a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f74cc099710 tx=0x7f74cc00cee0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.948+0000 7f74beffd640 1 -- 172.21.15.47:0/95158149 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8040070 con 0x55c24c9edf40 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.948+0000 7f74beffd640 1 -- 172.21.15.47:0/95158149 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c8004030 con 0x55c24c9edf40 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.948+0000 7f74beffd640 1 -- 172.21.15.47:0/95158149 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8035040 con 0x55c24c9edf40 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.949+0000 7f74dd790740 1 -- 172.21.15.47:0/95158149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 msgr2=0x55c24c9f82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.949+0000 7f74dd790740 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f82c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f74c802a940 tx=0x7f74c8031bc0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.949+0000 7f74dd790740 1 -- 172.21.15.47:0/95158149 shutdown_connections 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.949+0000 7f74dd790740 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.949+0000 7f74dd790740 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f82c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.949+0000 7f74dd790740 1 --2- 172.21.15.47:0/95158149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.949+0000 7f74dd790740 1 -- 172.21.15.47:0/95158149 >> 172.21.15.47:0/95158149 conn(0x55c24c9e9b30 msgr2=0x55c24c9e9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.950+0000 7f74dd790740 1 -- 172.21.15.47:0/95158149 shutdown_connections 2024-04-25T21:03:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.950+0000 7f74dd790740 1 -- 172.21.15.47:0/95158149 wait complete. 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.950+0000 7f74dd790740 1 -- start start 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.950+0000 7f74dd790740 1 Processor -- start 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74dd790740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74dd790740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74dd790740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74dd790740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c24c9fb3d0 con 0x55c24c9edf40 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74dd790740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c24c8ed0b0 con 0x55c24c9ed620 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74dd790740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c24c9fb550 con 0x55c24c9f8800 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74d8818640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74d8818640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57568/0 (socket says 172.21.15.47:57568) 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74d8818640 1 -- 172.21.15.47:0/2842017094 learned_addr learned my addr 172.21.15.47:0/2842017094 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:14.953 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] splice2pipe 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74dd790740 10 client.0 _ll_register_callbacks cb 0x55c24c8dad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.951+0000 7f74d3fff640 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74d3fff640 1 -- 172.21.15.47:0/2842017094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 msgr2=0x55c24c9f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74d3fff640 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74d3fff640 1 -- 172.21.15.47:0/2842017094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 msgr2=0x55c24c9f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74d3fff640 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74d3fff640 1 -- 172.21.15.47:0/2842017094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24c731620 con 0x55c24c9ed620 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74d8818640 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74d3fff640 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9f8170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c24c9eacc0 tx=0x7f74c0010da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2311de4647e8b981 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74b3fff640 1 -- 172.21.15.47:0/2842017094 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c000d9a0 con 0x55c24c9ed620 2024-04-25T21:03:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74b3fff640 1 -- 172.21.15.47:0/2842017094 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f74c000db40 con 0x55c24c9ed620 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74dd790740 1 -- 172.21.15.47:0/2842017094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c8004030 con 0x55c24c9ed620 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74b3fff640 1 -- 172.21.15.47:0/2842017094 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74c000dd40 con 0x55c24c9ed620 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74dd790740 10 client.25301 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.952+0000 7f74dd790740 1 -- 172.21.15.47:0/2842017094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c24ca07320 con 0x55c24c9ed620 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74b3fff640 1 -- 172.21.15.47:0/2842017094 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c8004030 con 0x55c24c9ed620 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74b3fff640 1 -- 172.21.15.47:0/2842017094 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f74c00048c0 con 0x55c24c9ed620 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74dd790740 10 client.25301 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74dd790740 10 client.25301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74dd790740 20 client.25301 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74dd790740 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c24ca08830 0x55c24ca0ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74dd790740 4 client.25301 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74dd790740 1 -- 172.21.15.47:0/2842017094 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c24c960390 con 0x55c24ca08830 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.953+0000 7f74d8818640 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c24ca08830 0x55c24ca0ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.954+0000 7f74b3fff640 10 client.25301 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.954+0000 7f74d8818640 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c24ca08830 0x55c24ca0ac60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c947d15c6ee83931 server_cookie=297270518a91ac1 in_seq=0 out_seq=0 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.954+0000 7f74b3fff640 1 -- 172.21.15.47:0/2842017094 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c24c960390 con 0x55c24ca08830 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.954+0000 7f74b3fff640 10 client.25301 handle_command_reply: tid=0 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.954+0000 7f74dd790740 1 client.25301 shutdown 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 -- 172.21.15.47:0/2842017094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 msgr2=0x55c24c9f8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9f8170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c24c9eacc0 tx=0x7f74c0010da0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 -- 172.21.15.47:0/2842017094 shutdown_connections 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c24ca08830 0x55c24ca0ac60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74b3fff640 0 client.25301 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 --2- 172.21.15.47:0/2842017094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24c9f8170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 -- 172.21.15.47:0/2842017094 >> 172.21.15.47:0/2842017094 conn(0x55c24c9e9b30 msgr2=0x55c24c9ef8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 -- 172.21.15.47:0/2842017094 shutdown_connections 2024-04-25T21:03:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 1 -- 172.21.15.47:0/2842017094 wait complete. 2024-04-25T21:03:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.955+0000 7f74dd790740 20 client.25301 trim_cache size 0 max 16384 2024-04-25T21:03:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74dd790740 1 Processor -- start 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74dd790740 1 -- start start 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74dd790740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9ed620 0x55c24ca04290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74dd790740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9edf40 0x55c24c9f5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74dd790740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9f8800 0x55c24c9f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74dd790740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41a5b30 con 0x55c24c9f8800 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74dd790740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41a33a0 con 0x55c24c9edf40 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74dd790740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41a1210 con 0x55c24c9ed620 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9f8800 0x55c24c9f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9f8800 0x55c24c9f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57582/0 (socket says 172.21.15.47:57582) 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74d3fff640 1 -- 172.21.15.47:0/4277342857 learned_addr learned my addr 172.21.15.47:0/4277342857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74d8818640 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9edf40 0x55c24c9f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.987+0000 7f74d3fff640 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9ed620 0x55c24ca04290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74d3fff640 1 -- 172.21.15.47:0/4277342857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9ed620 msgr2=0x55c24ca04290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74d3fff640 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9ed620 0x55c24ca04290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74d3fff640 1 -- 172.21.15.47:0/4277342857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9edf40 msgr2=0x55c24c9f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74d3fff640 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9edf40 0x55c24c9f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74d3fff640 1 -- 172.21.15.47:0/4277342857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c24c9ead60 con 0x55c24c9f8800 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74d3fff640 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9f8800 0x55c24c9f7ca0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f74c0001eb0 tx=0x7f74c0012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad232a1b0ab8bb97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74be7fc640 1 -- 172.21.15.47:0/4277342857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c002d040 con 0x55c24c9f8800 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74be7fc640 1 -- 172.21.15.47:0/4277342857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c00053b0 con 0x55c24c9f8800 2024-04-25T21:03:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.988+0000 7f74be7fc640 1 -- 172.21.15.47:0/4277342857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0029660 con 0x55c24c9f8800 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 -- 172.21.15.47:0/4277342857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9f8800 msgr2=0x55c24c9f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9f8800 0x55c24c9f7ca0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f74c0001eb0 tx=0x7f74c0012040 comp rx=0 tx=0).stop 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 -- 172.21.15.47:0/4277342857 shutdown_connections 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9f8800 0x55c24c9f7ca0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9edf40 0x55c24c9f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 --2- 172.21.15.47:0/4277342857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9ed620 0x55c24ca04290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 -- 172.21.15.47:0/4277342857 >> 172.21.15.47:0/4277342857 conn(0x55c24c9e9b30 msgr2=0x55c24c9f0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 -- 172.21.15.47:0/4277342857 shutdown_connections 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.989+0000 7f74dd790740 1 -- 172.21.15.47:0/4277342857 wait complete. 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 1 -- start start 2024-04-25T21:03:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 1 Processor -- start 2024-04-25T21:03:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24ca00990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41a1860 con 0x55c24c9edf40 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41a2cd0 con 0x55c24c9ed620 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41a1f30 con 0x55c24c9f8800 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74dd790740 10 client.0 _ll_register_callbacks cb 0x7f74c0004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74d8818640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.990+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24ca00990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24ca00990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:47938/0 (socket says 172.21.15.47:47938) 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 -- 172.21.15.47:0/3344204048 learned_addr learned my addr 172.21.15.47:0/3344204048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 -- 172.21.15.47:0/3344204048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 msgr2=0x55c24c9f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 -- 172.21.15.47:0/3344204048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 msgr2=0x55c24c9f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 -- 172.21.15.47:0/3344204048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24c9ee8e0 con 0x55c24c9ed620 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74d3fff640 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24ca00990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74c000f3d0 tx=0x7f74c0004940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf370f37b2f1783 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74b3fff640 1 -- 172.21.15.47:0/3344204048 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c002d040 con 0x55c24c9ed620 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74b3fff640 1 -- 172.21.15.47:0/3344204048 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f74c0005560 con 0x55c24c9ed620 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.991+0000 7f74dd790740 1 -- 172.21.15.47:0/3344204048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c00053b0 con 0x55c24c9ed620 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74b3fff640 1 -- 172.21.15.47:0/3344204048 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74c000d9a0 con 0x55c24c9ed620 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74dd790740 10 client.25307 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74dd790740 1 -- 172.21.15.47:0/3344204048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c24ca6e910 con 0x55c24c9ed620 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74b3fff640 1 -- 172.21.15.47:0/3344204048 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c000db40 con 0x55c24c9ed620 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74b3fff640 1 -- 172.21.15.47:0/3344204048 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f74c0033070 con 0x55c24c9ed620 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74dd790740 10 client.25307 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74dd790740 10 client.25307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74dd790740 20 client.25307 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74dd790740 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c24ca6f010 0x55c24ca71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74dd790740 4 client.25307 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:03:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.992+0000 7f74dd790740 1 -- 172.21.15.47:0/3344204048 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f74c00053b0 con 0x55c24ca6f010 2024-04-25T21:03:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.993+0000 7f74d8818640 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c24ca6f010 0x55c24ca71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.993+0000 7f74b3fff640 10 client.25307 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:14.993+0000 7f74d8818640 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c24ca6f010 0x55c24ca71430 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c083c535142963b server_cookie=b73b5a81567b3713 in_seq=0 out_seq=0 2024-04-25T21:03:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74b3fff640 1 -- 172.21.15.47:0/3344204048 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f74c00053b0 con 0x55c24ca6f010 2024-04-25T21:03:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74b3fff640 10 client.25307 handle_command_reply: tid=0 2024-04-25T21:03:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 client.25307 shutdown 2024-04-25T21:03:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 -- 172.21.15.47:0/3344204048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 msgr2=0x55c24ca00990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24ca00990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74c000f3d0 tx=0x7f74c0004940 comp rx=0 tx=0).stop 2024-04-25T21:03:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 -- 172.21.15.47:0/3344204048 shutdown_connections 2024-04-25T21:03:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55c24ca6f010 0x55c24ca71430 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c24c9f8800 0x55c24c9f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74b3fff640 0 client.25307 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c24c9edf40 0x55c24c9f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 --2- 172.21.15.47:0/3344204048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c24c9ed620 0x55c24ca00990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.002+0000 7f74dd790740 1 -- 172.21.15.47:0/3344204048 >> 172.21.15.47:0/3344204048 conn(0x55c24c9e9b30 msgr2=0x55c24c9e8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.003+0000 7f74dd790740 1 -- 172.21.15.47:0/3344204048 shutdown_connections 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.003+0000 7f74dd790740 1 -- 172.21.15.47:0/3344204048 wait complete. 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.003+0000 7f74dd790740 20 client.25307 trim_cache size 0 max 16384 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:03:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:03:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:03:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:03:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.003+0000 7f74daa7c640 1 -- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f74c40a3e40 msgr2=0x7f74c40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.003+0000 7f74daa7c640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f74c40a3e40 0x7f74c40a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f74cc099710 tx=0x7f74cc00cee0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 -- 172.21.15.47:0/2644349322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d4106f30 msgr2=0x7f74d406c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d4106f30 0x7f74d406c960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00eb10 tx=0x7f74cc00c2d0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 -- 172.21.15.47:0/2644349322 shutdown_connections 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f74c40a3e40 0x7f74c40a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d4106f30 0x7f74d406c960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d4100e10 0x7f74d4073c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 --2- 172.21.15.47:0/2644349322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d4100450 0x7f74d4073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 -- 172.21.15.47:0/2644349322 >> 172.21.15.47:0/2644349322 conn(0x7f74d40773e0 msgr2=0x7f74d40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 -- 172.21.15.47:0/2644349322 shutdown_connections 2024-04-25T21:03:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:15.004+0000 7f74daa7c640 1 -- 172.21.15.47:0/2644349322 wait complete. 2024-04-25T21:03:15.011 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] uuid_ioctl 2024-04-25T21:03:15.080 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-25T21:03:15.119 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fiemap-fault 2024-04-25T21:03:15.177 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fiemap-tester 2024-04-25T21:03:15.254 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_stripealign 2024-04-25T21:03:15.293 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] open_by_handle 2024-04-25T21:03:15.347 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_mmap_dio 2024-04-25T21:03:15.412 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] xfsfind 2024-04-25T21:03:15.482 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] t_reflink_read_race 2024-04-25T21:03:15.505 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] fssum 2024-04-25T21:03:15.579 INFO:tasks.workunit.client.0.smithi047.stdout:Building log-writes 2024-04-25T21:03:15.584 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] replay-log 2024-04-25T21:03:15.628 INFO:tasks.workunit.client.0.smithi047.stdout:Building perf 2024-04-25T21:03:15.631 INFO:tasks.workunit.client.0.smithi047.stdout:Building aio-dio-regress 2024-04-25T21:03:15.638 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aiocp 2024-04-25T21:03:15.698 INFO:tasks.workunit.client.0.smithi047.stdout:Building vfs 2024-04-25T21:03:15.704 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] vfstest 2024-04-25T21:03:15.880 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-25T21:03:15.947 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-append-write-read-race 2024-04-25T21:03:15.983 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-cow-race 2024-04-25T21:03:16.077 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-cycle-write 2024-04-25T21:03:16.121 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-eof-race 2024-04-25T21:03:16.154 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-extend-stat 2024-04-25T21:03:16.267 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-fcntl-race 2024-04-25T21:03:16.310 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-hole-filling-race 2024-04-25T21:03:16.317 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-invalidate-failure 2024-04-25T21:03:16.429 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-invalidate-readahead 2024-04-25T21:03:16.468 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aiodio_sparse2 2024-04-25T21:03:16.475 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-subblock-eof-read 2024-04-25T21:03:16.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:16 smithi047 ceph-mon[29195]: pgmap v101: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 251 op/s 2024-04-25T21:03:16.616 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-dio-write-verify 2024-04-25T21:03:16.620 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-25T21:03:16.704 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-25T21:03:16.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:16 smithi186 ceph-mon[32529]: pgmap v101: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 251 op/s 2024-04-25T21:03:16.763 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] aio-last-ref-held-by-io 2024-04-25T21:03:16.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:16 smithi092 ceph-mon[32524]: pgmap v101: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 251 op/s 2024-04-25T21:03:16.817 INFO:tasks.workunit.client.0.smithi047.stdout: [CC] mount-idmapped 2024-04-25T21:03:17.782 INFO:tasks.workunit.client.0.smithi047.stderr:+ cd .. 2024-04-25T21:03:17.782 INFO:tasks.workunit.client.0.smithi047.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-25T21:03:17.785 INFO:tasks.workunit.client.0.smithi047.stderr:+ OPTIONS=-z 2024-04-25T21:03:17.785 INFO:tasks.workunit.client.0.smithi047.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-25T21:03:17.785 INFO:tasks.workunit.client.0.smithi047.stdout:Seed set to 1 2024-04-25T21:03:17.812 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T21:03:17.812 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T21:03:17.814 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T21:03:17.814 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T21:03:17.814 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T21:03:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T21:03:17.820 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T21:03:17.820 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero size read 2024-04-25T21:03:17.820 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero size read 2024-04-25T21:03:17.820 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x81d54 2024-04-25T21:03:17.826 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xb86d7 2024-04-25T21:03:17.834 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xe627e 2024-04-25T21:03:17.875 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0xf3194 2024-04-25T21:03:17.900 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xf3ceb 2024-04-25T21:03:17.906 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0xf47e4 2024-04-25T21:03:18.004 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xfaccc 2024-04-25T21:03:18.399 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xfe423 2024-04-25T21:03:18.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:18 smithi047 ceph-mon[29195]: pgmap v102: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 403 KiB/s wr, 54 op/s 2024-04-25T21:03:18.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:18 smithi186 ceph-mon[32529]: pgmap v102: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 403 KiB/s wr, 54 op/s 2024-04-25T21:03:18.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:18 smithi092 ceph-mon[32524]: pgmap v102: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 403 KiB/s wr, 54 op/s 2024-04-25T21:03:20.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:20 smithi047 ceph-mon[29195]: pgmap v103: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 403 KiB/s wr, 54 op/s 2024-04-25T21:03:20.676 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xfe492 2024-04-25T21:03:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:20 smithi186 ceph-mon[32529]: pgmap v103: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 403 KiB/s wr, 54 op/s 2024-04-25T21:03:20.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:20 smithi092 ceph-mon[32524]: pgmap v103: 97 pgs: 97 active+clean; 58 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 403 KiB/s wr, 54 op/s 2024-04-25T21:03:22.493 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xff038 2024-04-25T21:03:22.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:22 smithi047 ceph-mon[29195]: pgmap v104: 97 pgs: 97 active+clean; 61 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-04-25T21:03:22.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:22 smithi186 ceph-mon[32529]: pgmap v104: 97 pgs: 97 active+clean; 61 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-04-25T21:03:22.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:22 smithi092 ceph-mon[32524]: pgmap v104: 97 pgs: 97 active+clean; 61 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-04-25T21:03:23.273 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0xffcac 2024-04-25T21:03:23.383 INFO:tasks.workunit.client.0.smithi047.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-25T21:03:24.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:24 smithi047 ceph-mon[29195]: pgmap v105: 97 pgs: 97 active+clean; 61 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-25T21:03:24.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:24 smithi186 ceph-mon[32529]: pgmap v105: 97 pgs: 97 active+clean; 61 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-25T21:03:24.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:24 smithi092 ceph-mon[32524]: pgmap v105: 97 pgs: 97 active+clean; 61 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-25T21:03:26.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:26 smithi047 ceph-mon[29195]: pgmap v106: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 511 op/s 2024-04-25T21:03:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:26 smithi186 ceph-mon[32529]: pgmap v106: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 511 op/s 2024-04-25T21:03:26.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:26 smithi092 ceph-mon[32524]: pgmap v106: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 511 op/s 2024-04-25T21:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 -- 172.21.15.47:0/399747735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41010f0 msgr2=0x7fb7e41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 --2- 172.21.15.47:0/399747735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41010f0 0x7fb7e41014d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009ca0 tx=0x7fb7dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 -- 172.21.15.47:0/399747735 shutdown_connections 2024-04-25T21:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 --2- 172.21.15.47:0/399747735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e4106170 0x7fb7e410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 --2- 172.21.15.47:0/399747735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e4101aa0 0x7fb7e4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 --2- 172.21.15.47:0/399747735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41010f0 0x7fb7e41014d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 -- 172.21.15.47:0/399747735 >> 172.21.15.47:0/399747735 conn(0x7fb7e4077fd0 msgr2=0x7fb7e40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 -- 172.21.15.47:0/399747735 shutdown_connections 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 -- 172.21.15.47:0/399747735 wait complete. 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.102+0000 7fb7eadef640 1 Processor -- start 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7eadef640 1 -- start start 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7eadef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e41010f0 0x7fb7e4110620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7eadef640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e4106170 0x7fb7e4110b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7eadef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41110a0 0x7fb7e41a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7eadef640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e406a190 con 0x7fb7e41110a0 2024-04-25T21:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7eadef640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e406a010 con 0x7fb7e41010f0 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7eadef640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e406a310 con 0x7fb7e4106170 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e938c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41110a0 0x7fb7e41a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e4106170 0x7fb7e4110b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e4106170 0x7fb7e4110b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:57362/0 (socket says 172.21.15.47:57362) 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e3fff640 1 -- 172.21.15.47:0/2890093035 learned_addr learned my addr 172.21.15.47:0/2890093035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e41010f0 0x7fb7e4110620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e938c640 1 -- 172.21.15.47:0/2890093035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e4106170 msgr2=0x7fb7e4110b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e938c640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e4106170 0x7fb7e4110b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e938c640 1 -- 172.21.15.47:0/2890093035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e41010f0 msgr2=0x7fb7e4110620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e938c640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e41010f0 0x7fb7e4110620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e938c640 1 -- 172.21.15.47:0/2890093035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e41a6f70 con 0x7fb7e41110a0 2024-04-25T21:03:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e938c640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41110a0 0x7fb7e41a6870 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00e970 tx=0x7fb7cc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.103+0000 7fb7e1ffb640 1 -- 172.21.15.47:0/2890093035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00ccc0 con 0x7fb7e41110a0 2024-04-25T21:03:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.104+0000 7fb7e1ffb640 1 -- 172.21.15.47:0/2890093035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7cc005020 con 0x7fb7e41110a0 2024-04-25T21:03:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.104+0000 7fb7eadef640 1 -- 172.21.15.47:0/2890093035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e41a7260 con 0x7fb7e41110a0 2024-04-25T21:03:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.104+0000 7fb7e1ffb640 1 -- 172.21.15.47:0/2890093035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc004a00 con 0x7fb7e41110a0 2024-04-25T21:03:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.104+0000 7fb7eadef640 1 -- 172.21.15.47:0/2890093035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e41a77b0 con 0x7fb7e41110a0 2024-04-25T21:03:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.106+0000 7fb7e1ffb640 1 -- 172.21.15.47:0/2890093035 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb7cc004ba0 con 0x7fb7e41110a0 2024-04-25T21:03:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.106+0000 7fb7edbd6740 1 Processor -- start 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.106+0000 7fb7edbd6740 1 -- start start 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.106+0000 7fb7e1ffb640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb7d00a3df0 0x7fb7d00a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.106+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06377a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed06384bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ed062dc970 con 0x55ed06377620 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ed062794a0 con 0x55ed06382800 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ed062d8930 con 0x55ed06377f40 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06377a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb7d00a3df0 0x7fb7d00a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e938c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed06384bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06377a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44302/0 (socket says 172.21.15.47:44302) 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 -- 172.21.15.47:0/140288796 learned_addr learned my addr 172.21.15.47:0/140288796 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e1ffb640 1 -- 172.21.15.47:0/2890093035 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fb7cc0d06d0 con 0x7fb7e41110a0 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 -- 172.21.15.47:0/140288796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 msgr2=0x55ed063822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 -- 172.21.15.47:0/140288796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 msgr2=0x55ed06384bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed06384bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 -- 172.21.15.47:0/140288796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed05fef210 con 0x55ed06377620 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e938c640 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed06384bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06377a00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc035000 tx=0x7fb7dc0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daefaae7edbf9020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.107+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb7d00a3df0 0x7fb7d00a62b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55ed06378bc0 tx=0x7fb7d4009a00 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7daffd640 1 -- 172.21.15.47:0/140288796 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc031cf0 con 0x55ed06377620 2024-04-25T21:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7daffd640 1 -- 172.21.15.47:0/140288796 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7dc036030 con 0x55ed06377620 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7daffd640 1 -- 172.21.15.47:0/140288796 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc004260 con 0x55ed06377620 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 -- 172.21.15.47:0/140288796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 msgr2=0x55ed06377a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06377a00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc035000 tx=0x7fb7dc0093f0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 -- 172.21.15.47:0/140288796 shutdown_connections 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed06384bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/140288796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06377a00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 -- 172.21.15.47:0/140288796 >> 172.21.15.47:0/140288796 conn(0x55ed06373b30 msgr2=0x55ed06373f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 -- 172.21.15.47:0/140288796 shutdown_connections 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.108+0000 7fb7edbd6740 1 -- 172.21.15.47:0/140288796 wait complete. 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 1 -- start start 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 1 Processor -- start 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06382150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed0637f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed0637f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ed06385250 con 0x55ed06377620 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ed062770b0 con 0x55ed06382800 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ed063853d0 con 0x55ed06377f40 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7edbd6740 10 client.0 _ll_register_callbacks cb 0x55ed06264d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7e8b8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06382150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed0637f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7e8b8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06382150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44314/0 (socket says 172.21.15.47:44314) 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7e8b8b640 1 -- 172.21.15.47:0/712512010 learned_addr learned my addr 172.21.15.47:0/712512010 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.109+0000 7fb7e938c640 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed0637f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7e3fff640 1 -- 172.21.15.47:0/712512010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 msgr2=0x55ed0637f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed0637f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7e3fff640 1 -- 172.21.15.47:0/712512010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 msgr2=0x55ed06382150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06382150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7e3fff640 1 -- 172.21.15.47:0/712512010 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed060bb620 con 0x55ed06377f40 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06382150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed0637f440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7e41120b0 tx=0x7fb7d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a82199c9cb7cfc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7c3fff640 1 -- 172.21.15.47:0/712512010 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4026070 con 0x55ed06377f40 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7edbd6740 1 -- 172.21.15.47:0/712512010 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7dc036030 con 0x55ed06377f40 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7c3fff640 1 -- 172.21.15.47:0/712512010 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fb7d4020450 con 0x55ed06377f40 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7c3fff640 1 -- 172.21.15.47:0/712512010 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb7d4009860 con 0x55ed06377f40 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7edbd6740 10 client.25315 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.110+0000 7fb7edbd6740 1 -- 172.21.15.47:0/712512010 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed063911d0 con 0x55ed06377f40 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7c3fff640 1 -- 172.21.15.47:0/712512010 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7dc036030 con 0x55ed06377f40 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7c3fff640 1 -- 172.21.15.47:0/712512010 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb7d401d040 con 0x55ed06377f40 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7edbd6740 10 client.25315 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7edbd6740 10 client.25315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7edbd6740 20 client.25315 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55ed06392690 0x55ed06394ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7edbd6740 4 client.25315 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7edbd6740 1 -- 172.21.15.47:0/712512010 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed062ea390 con 0x55ed06392690 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55ed06392690 0x55ed06394ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.111+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55ed06392690 0x55ed06394ae0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6fb4d7ffd79ffe server_cookie=42035f564ec3db2b in_seq=0 out_seq=0 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.112+0000 7fb7c3fff640 10 client.25315 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.112+0000 7fb7c3fff640 1 -- 172.21.15.47:0/712512010 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed062ea390 con 0x55ed06392690 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.112+0000 7fb7c3fff640 10 client.25315 handle_command_reply: tid=0 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.112+0000 7fb7edbd6740 1 client.25315 shutdown 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.112+0000 7fb7edbd6740 1 -- 172.21.15.47:0/712512010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 msgr2=0x55ed0637f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.112+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed0637f440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7e41120b0 tx=0x7fb7d4004730 comp rx=0 tx=0).stop 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 1 -- 172.21.15.47:0/712512010 shutdown_connections 2024-04-25T21:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55ed06392690 0x55ed06394ae0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06382800 0x55ed0637f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed0637f440 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/712512010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377620 0x55ed06382150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7c3fff640 0 client.25315 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 1 -- 172.21.15.47:0/712512010 >> 172.21.15.47:0/712512010 conn(0x55ed06373b30 msgr2=0x55ed06379860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 1 -- 172.21.15.47:0/712512010 shutdown_connections 2024-04-25T21:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 1 -- 172.21.15.47:0/712512010 wait complete. 2024-04-25T21:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.113+0000 7fb7edbd6740 20 client.25315 trim_cache size 0 max 16384 2024-04-25T21:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7edbd6740 1 Processor -- start 2024-04-25T21:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7edbd6740 1 -- start start 2024-04-25T21:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed0637aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377f40 0x55ed06372b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06382800 0x55ed06381c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41b2680 con 0x55ed06377f40 2024-04-25T21:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41afef0 con 0x55ed06377620 2024-04-25T21:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41add60 con 0x55ed06382800 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377f40 0x55ed06372b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377f40 0x55ed06372b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44326/0 (socket says 172.21.15.47:44326) 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e3fff640 1 -- 172.21.15.47:0/4000803948 learned_addr learned my addr 172.21.15.47:0/4000803948 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed0637aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e938c640 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06382800 0x55ed06381c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e3fff640 1 -- 172.21.15.47:0/4000803948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06382800 msgr2=0x55ed06381c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06382800 0x55ed06381c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e3fff640 1 -- 172.21.15.47:0/4000803948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 msgr2=0x55ed0637aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed0637aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.129+0000 7fb7e3fff640 1 -- 172.21.15.47:0/4000803948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed06395160 con 0x55ed06377f40 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377f40 0x55ed06372b30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb7d40097f0 tx=0x7fb7d4011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ad0df8862eec84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7da7fc640 1 -- 172.21.15.47:0/4000803948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4026040 con 0x55ed06377f40 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7da7fc640 1 -- 172.21.15.47:0/4000803948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7d40044f0 con 0x55ed06377f40 2024-04-25T21:03:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7da7fc640 1 -- 172.21.15.47:0/4000803948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d40035f0 con 0x55ed06377f40 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7edbd6740 1 -- 172.21.15.47:0/4000803948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377f40 msgr2=0x55ed06372b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377f40 0x55ed06372b30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb7d40097f0 tx=0x7fb7d4011630 comp rx=0 tx=0).stop 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7edbd6740 1 -- 172.21.15.47:0/4000803948 shutdown_connections 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06382800 0x55ed06381c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06377f40 0x55ed06372b30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/4000803948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed0637aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.130+0000 7fb7edbd6740 1 -- 172.21.15.47:0/4000803948 >> 172.21.15.47:0/4000803948 conn(0x55ed06373b30 msgr2=0x55ed06372420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 -- 172.21.15.47:0/4000803948 shutdown_connections 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 -- 172.21.15.47:0/4000803948 wait complete. 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 -- start start 2024-04-25T21:03:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 Processor -- start 2024-04-25T21:03:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed06386060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06382800 0x55ed06386ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41ae3b0 con 0x55ed06382800 2024-04-25T21:03:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41af820 con 0x55ed06377620 2024-04-25T21:03:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41aea80 con 0x55ed06377f40 2024-04-25T21:03:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7edbd6740 10 client.0 _ll_register_callbacks cb 0x7fb7d400fc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7e938c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06382800 0x55ed06386ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7e8b8b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed06386060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7e938c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06382800 0x55ed06386ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44336/0 (socket says 172.21.15.47:44336) 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.131+0000 7fb7e8b8b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed06386060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:60502/0 (socket says 172.21.15.47:60502) 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e938c640 1 -- 172.21.15.47:0/2459469371 learned_addr learned my addr 172.21.15.47:0/2459469371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e938c640 1 -- 172.21.15.47:0/2459469371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 msgr2=0x55ed063865a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e938c640 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063865a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e938c640 1 -- 172.21.15.47:0/2459469371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 msgr2=0x55ed06386060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e938c640 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed06386060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e938c640 1 -- 172.21.15.47:0/2459469371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed063785b0 con 0x55ed06382800 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e3fff640 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063865a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed06386060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7e938c640 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06382800 0x55ed06386ae0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc09c320 tx=0x7fb7cc09bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748fe8c6d5b8de55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7c3fff640 1 -- 172.21.15.47:0/2459469371 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc01c070 con 0x55ed06382800 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7c3fff640 1 -- 172.21.15.47:0/2459469371 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fb7cc0ce050 con 0x55ed06382800 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7edbd6740 1 -- 172.21.15.47:0/2459469371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d40044f0 con 0x55ed06382800 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7c3fff640 1 -- 172.21.15.47:0/2459469371 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb7cc0ce250 con 0x55ed06382800 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7edbd6740 10 client.15702 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7edbd6740 1 -- 172.21.15.47:0/2459469371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed063f8e10 con 0x55ed06382800 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.132+0000 7fb7c3fff640 1 -- 172.21.15.47:0/2459469371 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7d40044f0 con 0x55ed06382800 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7c3fff640 1 -- 172.21.15.47:0/2459469371 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb7cc09c880 con 0x55ed06382800 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7edbd6740 10 client.15702 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7edbd6740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7edbd6740 20 client.15702 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55ed063f95e0 0x55ed063fba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7edbd6740 4 client.15702 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:03:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7edbd6740 1 -- 172.21.15.47:0/2459469371 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed062e4200 con 0x55ed063f95e0 2024-04-25T21:03:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55ed063f95e0 0x55ed063fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7e8b8b640 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55ed063f95e0 0x55ed063fba00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734f7bd6015a0c62 server_cookie=92b4330c9e388923 in_seq=0 out_seq=0 2024-04-25T21:03:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.133+0000 7fb7c3fff640 10 client.15702 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7c3fff640 1 -- 172.21.15.47:0/2459469371 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed062e4200 con 0x55ed063f95e0 2024-04-25T21:03:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7c3fff640 10 client.15702 handle_command_reply: tid=0 2024-04-25T21:03:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 client.15702 shutdown 2024-04-25T21:03:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 -- 172.21.15.47:0/2459469371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06382800 msgr2=0x55ed06386ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06382800 0x55ed06386ae0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc09c320 tx=0x7fb7cc09bf80 comp rx=0 tx=0).stop 2024-04-25T21:03:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 -- 172.21.15.47:0/2459469371 shutdown_connections 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55ed063f95e0 0x55ed063fba00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed06382800 0x55ed06386ae0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed06377f40 0x55ed063865a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7c3fff640 0 client.15702 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 --2- 172.21.15.47:0/2459469371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed06377620 0x55ed06386060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 -- 172.21.15.47:0/2459469371 >> 172.21.15.47:0/2459469371 conn(0x55ed06373b30 msgr2=0x55ed0638d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 -- 172.21.15.47:0/2459469371 shutdown_connections 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.140+0000 7fb7edbd6740 1 -- 172.21.15.47:0/2459469371 wait complete. 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7edbd6740 20 client.15702 trim_cache size 0 max 16384 2024-04-25T21:03:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:03:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:03:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:03:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 -- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb7d00a3df0 msgr2=0x7fb7d00a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb7d00a3df0 0x7fb7d00a62b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55ed06378bc0 tx=0x7fb7d4009a00 comp rx=0 tx=0).stop 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 -- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41110a0 msgr2=0x7fb7e41a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41110a0 0x7fb7e41a6870 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00e970 tx=0x7fb7cc00ee30 comp rx=0 tx=0).stop 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 -- 172.21.15.47:0/2890093035 shutdown_connections 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e41110a0 0x7fb7e41a6870 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e4106170 0x7fb7e4110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb7d00a3df0 0x7fb7d00a62b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 --2- 172.21.15.47:0/2890093035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e41010f0 0x7fb7e4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 -- 172.21.15.47:0/2890093035 >> 172.21.15.47:0/2890093035 conn(0x7fb7e4077fd0 msgr2=0x7fb7e40fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 -- 172.21.15.47:0/2890093035 shutdown_connections 2024-04-25T21:03:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:28.141+0000 7fb7eadef640 1 -- 172.21.15.47:0/2890093035 wait complete. 2024-04-25T21:03:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:28 smithi047 ceph-mon[29195]: pgmap v107: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 505 op/s 2024-04-25T21:03:28.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:28 smithi186 ceph-mon[32529]: pgmap v107: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 505 op/s 2024-04-25T21:03:28.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:28 smithi092 ceph-mon[32524]: pgmap v107: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 505 op/s 2024-04-25T21:03:30.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:30 smithi047 ceph-mon[29195]: pgmap v108: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 505 op/s 2024-04-25T21:03:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:30 smithi186 ceph-mon[32529]: pgmap v108: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 505 op/s 2024-04-25T21:03:30.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:30 smithi092 ceph-mon[32524]: pgmap v108: 97 pgs: 97 active+clean; 64 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 505 op/s 2024-04-25T21:03:32.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:32 smithi047 ceph-mon[29195]: pgmap v109: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 817 op/s 2024-04-25T21:03:32.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:32 smithi186 ceph-mon[32529]: pgmap v109: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 817 op/s 2024-04-25T21:03:32.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:32 smithi092 ceph-mon[32524]: pgmap v109: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 817 op/s 2024-04-25T21:03:34.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:34 smithi047 ceph-mon[29195]: pgmap v110: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 659 op/s 2024-04-25T21:03:34.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:34 smithi186 ceph-mon[32529]: pgmap v110: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 659 op/s 2024-04-25T21:03:34.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:34 smithi092 ceph-mon[32524]: pgmap v110: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 659 op/s 2024-04-25T21:03:36.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:36 smithi047 ceph-mon[29195]: pgmap v111: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.2 MiB/s wr, 942 op/s 2024-04-25T21:03:36.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:36 smithi186 ceph-mon[32529]: pgmap v111: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.2 MiB/s wr, 942 op/s 2024-04-25T21:03:36.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:36 smithi092 ceph-mon[32524]: pgmap v111: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.2 MiB/s wr, 942 op/s 2024-04-25T21:03:36.828 INFO:tasks.workunit.client.0.smithi047.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-25T21:03:37.190 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xffcdd 2024-04-25T21:03:38.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:38 smithi047 ceph-mon[29195]: pgmap v112: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 594 op/s 2024-04-25T21:03:38.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:38 smithi186 ceph-mon[32529]: pgmap v112: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 594 op/s 2024-04-25T21:03:38.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:38 smithi092 ceph-mon[32524]: pgmap v112: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 594 op/s 2024-04-25T21:03:38.840 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xffd97 2024-04-25T21:03:40.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:40 smithi047 ceph-mon[29195]: pgmap v113: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 594 op/s 2024-04-25T21:03:40.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:40 smithi186 ceph-mon[32529]: pgmap v113: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 594 op/s 2024-04-25T21:03:40.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:40 smithi092 ceph-mon[32524]: pgmap v113: 97 pgs: 97 active+clean; 73 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 594 op/s 2024-04-25T21:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.235+0000 7f6420abb640 1 -- 172.21.15.47:0/3059582071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 msgr2=0x7f641c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.235+0000 7f6420abb640 1 --2- 172.21.15.47:0/3059582071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 0x7f641c10dd40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6414009ca0 tx=0x7f641402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.235+0000 7f6420abb640 1 -- 172.21.15.47:0/3059582071 shutdown_connections 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.236+0000 7f6420abb640 1 --2- 172.21.15.47:0/3059582071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 0x7f641c10dd40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.236+0000 7f6420abb640 1 --2- 172.21.15.47:0/3059582071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f641c101aa0 0x7f641c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.236+0000 7f6420abb640 1 --2- 172.21.15.47:0/3059582071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f641c1010f0 0x7f641c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.236+0000 7f6420abb640 1 -- 172.21.15.47:0/3059582071 >> 172.21.15.47:0/3059582071 conn(0x7f641c077fd0 msgr2=0x7f641c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.236+0000 7f6420abb640 1 -- 172.21.15.47:0/3059582071 shutdown_connections 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.236+0000 7f6420abb640 1 -- 172.21.15.47:0/3059582071 wait complete. 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.236+0000 7f6420abb640 1 Processor -- start 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.236+0000 7f6420abb640 1 -- start start 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6420abb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f641c1010f0 0x7f641c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6420abb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 0x7f641c197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6420abb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f641c197d50 0x7f641c1a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6420abb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1110e0 con 0x7f641c106170 2024-04-25T21:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6420abb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f641c110f60 con 0x7f641c197d50 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6420abb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f641c111260 con 0x7f641c1010f0 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 0x7f641c197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 0x7f641c197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34206/0 (socket says 172.21.15.47:34206) 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f641ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f641c197d50 0x7f641c1a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6419d9b640 1 -- 172.21.15.47:0/2209527433 learned_addr learned my addr 172.21.15.47:0/2209527433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f641a59c640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f641c1010f0 0x7f641c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f641ad9d640 1 -- 172.21.15.47:0/2209527433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f641c1010f0 msgr2=0x7f641c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f641ad9d640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f641c1010f0 0x7f641c1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f641ad9d640 1 -- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 msgr2=0x7f641c197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f641ad9d640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 0x7f641c197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f641ad9d640 1 -- 172.21.15.47:0/2209527433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c1a6f10 con 0x7f641c197d50 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6419d9b640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 0x7f641c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f641ad9d640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f641c197d50 0x7f641c1a6950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f641402ade0 tx=0x7f6414031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f640b7fe640 1 -- 172.21.15.47:0/2209527433 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414003b70 con 0x7f641c197d50 2024-04-25T21:03:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f6420abb640 1 -- 172.21.15.47:0/2209527433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c1a71a0 con 0x7f641c197d50 2024-04-25T21:03:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.237+0000 7f640b7fe640 1 -- 172.21.15.47:0/2209527433 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6414031910 con 0x7f641c197d50 2024-04-25T21:03:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.238+0000 7f640b7fe640 1 -- 172.21.15.47:0/2209527433 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641402f950 con 0x7f641c197d50 2024-04-25T21:03:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.238+0000 7f6420abb640 1 -- 172.21.15.47:0/2209527433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c1a76e0 con 0x7f641c197d50 2024-04-25T21:03:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f640b7fe640 1 -- 172.21.15.47:0/2209527433 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f641402faf0 con 0x7f641c197d50 2024-04-25T21:03:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f64237ce740 1 Processor -- start 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f64237ce740 1 -- start start 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f64237ce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f64237ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f64237ce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f64237ce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562db2407970 con 0x562db24ad800 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f64237ce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562db23a44a0 con 0x562db24a2620 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.240+0000 7f64237ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562db2403930 con 0x562db24a2f40 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34216/0 (socket says 172.21.15.47:34216) 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f6419d9b640 1 -- 172.21.15.47:0/1649668349 learned_addr learned my addr 172.21.15.47:0/1649668349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f640b7fe640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f64040a3dc0 0x7f64040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1649668349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641ad9d640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f64040a3dc0 0x7f64040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641a59c640 1 -- 172.21.15.47:0/1649668349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 msgr2=0x562db24a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641a59c640 1 --2- 172.21.15.47:0/1649668349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641a59c640 1 -- 172.21.15.47:0/1649668349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 msgr2=0x562db24afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641a59c640 1 --2- 172.21.15.47:0/1649668349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641a59c640 1 -- 172.21.15.47:0/1649668349 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db211a210 con 0x562db24a2f40 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f640b7fe640 1 -- 172.21.15.47:0/2209527433 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f64140f19c0 con 0x7f641c197d50 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641a59c640 1 --2- 172.21.15.47:0/1649668349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24ad2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f640c009c00 tx=0x7f640c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9365ce8fb423c85d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f641ad9d640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f64040a3dc0 0x7f64040a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64140b9e80 tx=0x7f641402a8c0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f6408ff9640 1 -- 172.21.15.47:0/1649668349 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c004050 con 0x562db24a2f40 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f6408ff9640 1 -- 172.21.15.47:0/1649668349 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f640c0054d0 con 0x562db24a2f40 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.241+0000 7f6408ff9640 1 -- 172.21.15.47:0/1649668349 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c004ce0 con 0x562db24a2f40 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 -- 172.21.15.47:0/1649668349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 msgr2=0x562db24ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 --2- 172.21.15.47:0/1649668349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24ad2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f640c009c00 tx=0x7f640c011810 comp rx=0 tx=0).stop 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 -- 172.21.15.47:0/1649668349 shutdown_connections 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 --2- 172.21.15.47:0/1649668349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 --2- 172.21.15.47:0/1649668349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24ad2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 --2- 172.21.15.47:0/1649668349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 -- 172.21.15.47:0/1649668349 >> 172.21.15.47:0/1649668349 conn(0x562db249eb30 msgr2=0x562db249ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 -- 172.21.15.47:0/1649668349 shutdown_connections 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.242+0000 7f64237ce740 1 -- 172.21.15.47:0/1649668349 wait complete. 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 1 -- start start 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 1 Processor -- start 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 0x562db24aa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562db24b0430 con 0x562db24a2f40 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562db23a20b0 con 0x562db24a2620 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562db24b05b0 con 0x562db24ad800 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f64237ce740 10 client.0 _ll_register_callbacks cb 0x562db238fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f641a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.243+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:34404/0 (socket says 172.21.15.47:34404) 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 -- 172.21.15.47:0/1728810878 learned_addr learned my addr 172.21.15.47:0/1728810878 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 0x562db24aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 -- 172.21.15.47:0/1728810878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 msgr2=0x562db24aa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 0x562db24aa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 -- 172.21.15.47:0/1728810878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 msgr2=0x562db24aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 -- 172.21.15.47:0/1728810878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db21e6620 con 0x562db24a2620 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f641a59c640 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24ad0e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6410011e60 tx=0x7f6410004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b77b7bd33a1731b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6401ffb640 1 -- 172.21.15.47:0/1728810878 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410026060 con 0x562db24a2620 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f64237ce740 1 -- 172.21.15.47:0/1728810878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c0054d0 con 0x562db24a2620 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6401ffb640 1 -- 172.21.15.47:0/1728810878 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6410004250 con 0x562db24a2620 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f6401ffb640 1 -- 172.21.15.47:0/1728810878 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6410004450 con 0x562db24a2620 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f64237ce740 10 client.25337 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.244+0000 7f64237ce740 1 -- 172.21.15.47:0/1728810878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562db24bc3c0 con 0x562db24a2620 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f6401ffb640 1 -- 172.21.15.47:0/1728810878 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f640c0054d0 con 0x562db24a2620 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f6401ffb640 1 -- 172.21.15.47:0/1728810878 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6410003820 con 0x562db24a2620 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f64237ce740 10 client.25337 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f64237ce740 10 client.25337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f64237ce740 20 client.25337 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f64237ce740 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562db24bd830 0x562db24bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f64237ce740 4 client.25337 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f64237ce740 1 -- 172.21.15.47:0/1728810878 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562db2415390 con 0x562db24bd830 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.245+0000 7f641a59c640 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562db24bd830 0x562db24bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f641a59c640 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562db24bd830 0x562db24bfc40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f32e281a69e0f2bd server_cookie=dc57b29191bdc6b6 in_seq=0 out_seq=0 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f6401ffb640 10 client.25337 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f6401ffb640 1 -- 172.21.15.47:0/1728810878 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562db2415390 con 0x562db24bd830 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f6401ffb640 10 client.25337 handle_command_reply: tid=0 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 client.25337 shutdown 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 -- 172.21.15.47:0/1728810878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 msgr2=0x562db24ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24ad0e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6410011e60 tx=0x7f6410004730 comp rx=0 tx=0).stop 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 -- 172.21.15.47:0/1728810878 shutdown_connections 2024-04-25T21:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562db24bd830 0x562db24bfc40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 0x562db24aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f6401ffb640 0 client.25337 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 --2- 172.21.15.47:0/1728810878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24ad0e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 -- 172.21.15.47:0/1728810878 >> 172.21.15.47:0/1728810878 conn(0x562db249eb30 msgr2=0x562db24a4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.246+0000 7f64237ce740 1 -- 172.21.15.47:0/1728810878 shutdown_connections 2024-04-25T21:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.247+0000 7f64237ce740 1 -- 172.21.15.47:0/1728810878 wait complete. 2024-04-25T21:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.247+0000 7f64237ce740 20 client.25337 trim_cache size 0 max 16384 2024-04-25T21:03:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.261+0000 7f64237ce740 1 Processor -- start 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.261+0000 7f64237ce740 1 -- start start 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.261+0000 7f64237ce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.261+0000 7f64237ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.261+0000 7f64237ce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.261+0000 7f64237ce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1b25a0 con 0x562db24ad800 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.261+0000 7f64237ce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1afe10 con 0x562db24a2620 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.261+0000 7f64237ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1adc80 con 0x562db24a2f40 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:53226/0 (socket says 172.21.15.47:53226) 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 -- 172.21.15.47:0/469552279 learned_addr learned my addr 172.21.15.47:0/469552279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 -- 172.21.15.47:0/469552279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 msgr2=0x562db24b92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 --2- 172.21.15.47:0/469552279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 -- 172.21.15.47:0/469552279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 msgr2=0x562db24acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 --2- 172.21.15.47:0/469552279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 -- 172.21.15.47:0/469552279 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db2499090 con 0x562db24a2f40 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f641a59c640 1 --2- 172.21.15.47:0/469552279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24a58c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f640c009990 tx=0x7f640c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f039b79079a1c9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f6403fff640 1 -- 172.21.15.47:0/469552279 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c032070 con 0x562db24a2f40 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f6403fff640 1 -- 172.21.15.47:0/469552279 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f640c0047c0 con 0x562db24a2f40 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.262+0000 7f6403fff640 1 -- 172.21.15.47:0/469552279 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0035f0 con 0x562db24a2f40 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 -- 172.21.15.47:0/469552279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 msgr2=0x562db24a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 --2- 172.21.15.47:0/469552279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24a58c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f640c009990 tx=0x7f640c005f90 comp rx=0 tx=0).stop 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 -- 172.21.15.47:0/469552279 shutdown_connections 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 --2- 172.21.15.47:0/469552279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24ad800 0x562db24acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 --2- 172.21.15.47:0/469552279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24a2f40 0x562db24a58c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 --2- 172.21.15.47:0/469552279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 -- 172.21.15.47:0/469552279 >> 172.21.15.47:0/469552279 conn(0x562db249eb30 msgr2=0x562db24a51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 -- 172.21.15.47:0/469552279 shutdown_connections 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 -- 172.21.15.47:0/469552279 wait complete. 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 -- start start 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.263+0000 7f64237ce740 1 Processor -- start 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f64237ce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f64237ce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f64237ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 0x562db24b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f64237ce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1ae2d0 con 0x562db24a2f40 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f64237ce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1af740 con 0x562db24a2620 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f64237ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1ae9a0 con 0x562db24ad800 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f64237ce740 10 client.0 _ll_register_callbacks cb 0x7f641000f830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f641a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:34420/0 (socket says 172.21.15.47:34420) 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 -- 172.21.15.47:0/1084809772 learned_addr learned my addr 172.21.15.47:0/1084809772 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 0x562db24b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 -- 172.21.15.47:0/1084809772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 msgr2=0x562db24b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 0x562db24b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 -- 172.21.15.47:0/1084809772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 msgr2=0x562db24b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 -- 172.21.15.47:0/1084809772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db24b1820 con 0x562db24a2620 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6419d9b640 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b58c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6410001eb0 tx=0x7f6410013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bace4e2591b6532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.264+0000 7f6401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f6401ffb640 1 -- 172.21.15.47:0/1084809772 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410003e60 con 0x562db24a2620 2024-04-25T21:03:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f6401ffb640 1 -- 172.21.15.47:0/1084809772 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6410004250 con 0x562db24a2620 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 1 -- 172.21.15.47:0/1084809772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562db240f200 con 0x562db24a2620 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f6401ffb640 1 -- 172.21.15.47:0/1084809772 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6410005020 con 0x562db24a2620 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 10 client.25343 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 1 -- 172.21.15.47:0/1084809772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562db2523ae0 con 0x562db24a2620 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f6401ffb640 1 -- 172.21.15.47:0/1084809772 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562db240f200 con 0x562db24a2620 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f6401ffb640 1 -- 172.21.15.47:0/1084809772 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6410004a70 con 0x562db24a2620 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 10 client.25343 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 20 client.25343 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562db24bd830 0x562db2526160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 4 client.25343 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f64237ce740 1 -- 172.21.15.47:0/1084809772 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562db238b5f0 con 0x562db24bd830 2024-04-25T21:03:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.265+0000 7f641a59c640 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562db24bd830 0x562db2526160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.266+0000 7f6401ffb640 10 client.25343 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.266+0000 7f641a59c640 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562db24bd830 0x562db2526160 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0fb0e2c00d54f09 server_cookie=a2c2cad93a1e61de in_seq=0 out_seq=0 2024-04-25T21:03:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.274+0000 7f6401ffb640 1 -- 172.21.15.47:0/1084809772 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562db238b5f0 con 0x562db24bd830 2024-04-25T21:03:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.274+0000 7f6401ffb640 10 client.25343 handle_command_reply: tid=0 2024-04-25T21:03:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.274+0000 7f64237ce740 1 client.25343 shutdown 2024-04-25T21:03:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 -- 172.21.15.47:0/1084809772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 msgr2=0x562db24b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b58c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6410001eb0 tx=0x7f6410013040 comp rx=0 tx=0).stop 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 -- 172.21.15.47:0/1084809772 shutdown_connections 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x562db24bd830 0x562db2526160 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562db24ad800 0x562db24b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6401ffb640 0 client.25343 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562db24a2f40 0x562db24b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 --2- 172.21.15.47:0/1084809772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db24a2620 0x562db24b58c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 -- 172.21.15.47:0/1084809772 >> 172.21.15.47:0/1084809772 conn(0x562db249eb30 msgr2=0x562db24a5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 -- 172.21.15.47:0/1084809772 shutdown_connections 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 1 -- 172.21.15.47:0/1084809772 wait complete. 2024-04-25T21:03:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f64237ce740 20 client.25343 trim_cache size 0 max 16384 2024-04-25T21:03:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:03:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:03:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:03:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 -- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f64040a3dc0 msgr2=0x7f64040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f64040a3dc0 0x7f64040a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64140b9e80 tx=0x7f641402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 -- 172.21.15.47:0/2209527433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f641c197d50 msgr2=0x7f641c1a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f641c197d50 0x7f641c1a6950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f641402ade0 tx=0x7f6414031660 comp rx=0 tx=0).stop 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 -- 172.21.15.47:0/2209527433 shutdown_connections 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f64040a3dc0 0x7f64040a6280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f641c197d50 0x7f641c1a6950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f641c106170 0x7f641c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 --2- 172.21.15.47:0/2209527433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f641c1010f0 0x7f641c1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 -- 172.21.15.47:0/2209527433 >> 172.21.15.47:0/2209527433 conn(0x7f641c077fd0 msgr2=0x7f641c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 -- 172.21.15.47:0/2209527433 shutdown_connections 2024-04-25T21:03:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:41.275+0000 7f6420abb640 1 -- 172.21.15.47:0/2209527433 wait complete. 2024-04-25T21:03:42.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:42 smithi186 ceph-mon[32529]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 852 op/s 2024-04-25T21:03:42.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:42 smithi092 ceph-mon[32524]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 852 op/s 2024-04-25T21:03:42.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:42 smithi047 ceph-mon[29195]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 852 op/s 2024-04-25T21:03:44.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:44 smithi186 ceph-mon[32529]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 540 op/s 2024-04-25T21:03:44.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:44 smithi092 ceph-mon[32524]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 540 op/s 2024-04-25T21:03:44.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:44 smithi047 ceph-mon[29195]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 540 op/s 2024-04-25T21:03:46.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:46 smithi186 ceph-mon[32529]: pgmap v116: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 773 op/s 2024-04-25T21:03:46.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:46 smithi092 ceph-mon[32524]: pgmap v116: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 773 op/s 2024-04-25T21:03:46.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:46 smithi047 ceph-mon[29195]: pgmap v116: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 773 op/s 2024-04-25T21:03:47.035 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0xffdb8 2024-04-25T21:03:48.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:48 smithi186 ceph-mon[32529]: pgmap v117: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-25T21:03:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:48 smithi092 ceph-mon[32524]: pgmap v117: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-25T21:03:48.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:48 smithi047 ceph-mon[29195]: pgmap v117: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-25T21:03:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:50 smithi186 ceph-mon[32529]: pgmap v118: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-25T21:03:50.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:50 smithi092 ceph-mon[32524]: pgmap v118: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-25T21:03:50.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:50 smithi047 ceph-mon[29195]: pgmap v118: 97 pgs: 97 active+clean; 83 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-25T21:03:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:52 smithi186 ceph-mon[32529]: pgmap v119: 97 pgs: 97 active+clean; 88 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 700 op/s 2024-04-25T21:03:52.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:52 smithi092 ceph-mon[32524]: pgmap v119: 97 pgs: 97 active+clean; 88 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 700 op/s 2024-04-25T21:03:52.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:52 smithi047 ceph-mon[29195]: pgmap v119: 97 pgs: 97 active+clean; 88 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 700 op/s 2024-04-25T21:03:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.366+0000 7fee81a49640 1 -- 172.21.15.47:0/332121099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 msgr2=0x7fee7c10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 --2- 172.21.15.47:0/332121099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 0x7fee7c10ee60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fee74009ca0 tx=0x7fee7402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 -- 172.21.15.47:0/332121099 shutdown_connections 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 --2- 172.21.15.47:0/332121099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 0x7fee7c10ee60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 --2- 172.21.15.47:0/332121099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee7c102b70 0x7fee7c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 --2- 172.21.15.47:0/332121099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee7c1021c0 0x7fee7c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 -- 172.21.15.47:0/332121099 >> 172.21.15.47:0/332121099 conn(0x7fee7c0fda20 msgr2=0x7fee7c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 -- 172.21.15.47:0/332121099 shutdown_connections 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 -- 172.21.15.47:0/332121099 wait complete. 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 Processor -- start 2024-04-25T21:03:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.367+0000 7fee81a49640 1 -- start start 2024-04-25T21:03:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee81a49640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee7c1021c0 0x7fee7c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee81a49640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 0x7fee7c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee81a49640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee7c074cd0 0x7fee7c1a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee81a49640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1121e0 con 0x7fee7c107290 2024-04-25T21:03:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee81a49640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c112060 con 0x7fee7c1021c0 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee81a49640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c112360 con 0x7fee7c074cd0 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee73fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 0x7fee7c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee7c1021c0 0x7fee7c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee73fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 0x7fee7c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50898/0 (socket says 172.21.15.47:50898) 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee7c1021c0 0x7fee7c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:60842/0 (socket says 172.21.15.47:60842) 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee73fff640 1 -- 172.21.15.47:0/3235082938 learned_addr learned my addr 172.21.15.47:0/3235082938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee7c074cd0 0x7fee7c1a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7affd640 1 -- 172.21.15.47:0/3235082938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee7c074cd0 msgr2=0x7fee7c1a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7affd640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee7c074cd0 0x7fee7c1a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7affd640 1 -- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 msgr2=0x7fee7c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7affd640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 0x7fee7c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7affd640 1 -- 172.21.15.47:0/3235082938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c1a6e90 con 0x7fee7c1021c0 2024-04-25T21:03:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee73fff640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 0x7fee7c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee7affd640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee7c1021c0 0x7fee7c074250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c970 tx=0x7fee6c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.368+0000 7fee80a47640 1 -- 172.21.15.47:0/3235082938 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c007bb0 con 0x7fee7c1021c0 2024-04-25T21:03:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.369+0000 7fee81a49640 1 -- 172.21.15.47:0/3235082938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c1a7180 con 0x7fee7c1021c0 2024-04-25T21:03:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.369+0000 7fee80a47640 1 -- 172.21.15.47:0/3235082938 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6c007d50 con 0x7fee7c1021c0 2024-04-25T21:03:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.369+0000 7fee80a47640 1 -- 172.21.15.47:0/3235082938 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c018780 con 0x7fee7c1021c0 2024-04-25T21:03:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.369+0000 7fee81a49640 1 -- 172.21.15.47:0/3235082938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c1a76d0 con 0x7fee7c1021c0 2024-04-25T21:03:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.371+0000 7fee80a47640 1 -- 172.21.15.47:0/3235082938 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fee6c00f450 con 0x7fee7c1021c0 2024-04-25T21:03:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.371+0000 7fee84833740 1 Processor -- start 2024-04-25T21:03:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.371+0000 7fee84833740 1 -- start start 2024-04-25T21:03:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee80a47640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fee680a3dc0 0x7fee680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee84833740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cdd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee84833740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee84833740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee84833740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55af9cd3e970 con 0x55af9cde4800 2024-04-25T21:03:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee84833740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55af9ccdb4a0 con 0x55af9cdd9620 2024-04-25T21:03:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee84833740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55af9cd3a930 con 0x55af9cdd9f40 2024-04-25T21:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee80a47640 1 -- 172.21.15.47:0/3235082938 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fee6c0d07e0 con 0x7fee7c1021c0 2024-04-25T21:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fee680a3dc0 0x7fee680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:48716/0 (socket says 172.21.15.47:48716) 2024-04-25T21:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 -- 172.21.15.47:0/1883264291 learned_addr learned my addr 172.21.15.47:0/1883264291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee7affd640 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cdd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 -- 172.21.15.47:0/1883264291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 msgr2=0x55af9cdd9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cdd9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 -- 172.21.15.47:0/1883264291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 msgr2=0x55af9cde6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 -- 172.21.15.47:0/1883264291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9ca51210 con 0x55af9cdd9f40 2024-04-25T21:03:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee7affd640 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cdd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fee680a3dc0 0x7fee680a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee7402f500 tx=0x7fee74032c20 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.372+0000 7fee73fff640 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde42c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee64011510 tx=0x7fee640119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b17ab24d182c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee71ffb640 1 -- 172.21.15.47:0/1883264291 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6401e070 con 0x55af9cdd9f40 2024-04-25T21:03:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee71ffb640 1 -- 172.21.15.47:0/1883264291 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6400fa70 con 0x55af9cdd9f40 2024-04-25T21:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee71ffb640 1 -- 172.21.15.47:0/1883264291 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64019b80 con 0x55af9cdd9f40 2024-04-25T21:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee84833740 1 -- 172.21.15.47:0/1883264291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 msgr2=0x55af9cde42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee84833740 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde42c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee64011510 tx=0x7fee640119d0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee84833740 1 -- 172.21.15.47:0/1883264291 shutdown_connections 2024-04-25T21:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee84833740 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee84833740 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde42c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee84833740 1 --2- 172.21.15.47:0/1883264291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cdd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.375+0000 7fee84833740 1 -- 172.21.15.47:0/1883264291 >> 172.21.15.47:0/1883264291 conn(0x55af9cdd5b30 msgr2=0x55af9cdd5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 -- 172.21.15.47:0/1883264291 shutdown_connections 2024-04-25T21:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 -- 172.21.15.47:0/1883264291 wait complete. 2024-04-25T21:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 -- start start 2024-04-25T21:03:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 Processor -- start 2024-04-25T21:03:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cde4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55af9cde7430 con 0x55af9cde4800 2024-04-25T21:03:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55af9ccd90b0 con 0x55af9cdd9620 2024-04-25T21:03:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55af9cde75b0 con 0x55af9cdd9f40 2024-04-25T21:03:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee84833740 10 client.0 _ll_register_callbacks cb 0x55af9ccc6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee7b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee7b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50922/0 (socket says 172.21.15.47:50922) 2024-04-25T21:03:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.376+0000 7fee7b7fe640 1 -- 172.21.15.47:0/4012025959 learned_addr learned my addr 172.21.15.47:0/4012025959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee73fff640 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee7affd640 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cde4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee7b7fe640 1 -- 172.21.15.47:0/4012025959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 msgr2=0x55af9cde1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee7b7fe640 1 -- 172.21.15.47:0/4012025959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 msgr2=0x55af9cde4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cde4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee7b7fe640 1 -- 172.21.15.47:0/4012025959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9cb1d620 con 0x55af9cde4800 2024-04-25T21:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee73fff640 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde1c60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fee74035000 tx=0x7fee74047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9adb0e2edbd0c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee5affd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee5affd640 1 -- 172.21.15.47:0/4012025959 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7404f070 con 0x55af9cde4800 2024-04-25T21:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee84833740 1 -- 172.21.15.47:0/4012025959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6400fa70 con 0x55af9cde4800 2024-04-25T21:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee5affd640 1 -- 172.21.15.47:0/4012025959 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fee74031740 con 0x55af9cde4800 2024-04-25T21:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee5affd640 1 -- 172.21.15.47:0/4012025959 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee74004b40 con 0x55af9cde4800 2024-04-25T21:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee84833740 10 client.15732 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.377+0000 7fee84833740 1 -- 172.21.15.47:0/4012025959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af9cdf3460 con 0x55af9cde4800 2024-04-25T21:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee5affd640 1 -- 172.21.15.47:0/4012025959 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6400fa70 con 0x55af9cde4800 2024-04-25T21:03:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee5affd640 1 -- 172.21.15.47:0/4012025959 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fee74004310 con 0x55af9cde4800 2024-04-25T21:03:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee84833740 10 client.15732 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee84833740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee84833740 20 client.15732 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee84833740 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55af9cdf48d0 0x55af9cdf6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee84833740 4 client.15732 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:03:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee84833740 1 -- 172.21.15.47:0/4012025959 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af9cd4c390 con 0x55af9cdf48d0 2024-04-25T21:03:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.378+0000 7fee7affd640 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55af9cdf48d0 0x55af9cdf6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee7affd640 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55af9cdf48d0 0x55af9cdf6ce0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58d9187bc649e994 server_cookie=766117bfe739df2c in_seq=0 out_seq=0 2024-04-25T21:03:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee5affd640 10 client.15732 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee5affd640 1 -- 172.21.15.47:0/4012025959 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af9cd4c390 con 0x55af9cdf48d0 2024-04-25T21:03:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee5affd640 10 client.15732 handle_command_reply: tid=0 2024-04-25T21:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 client.15732 shutdown 2024-04-25T21:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 -- 172.21.15.47:0/4012025959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 msgr2=0x55af9cde1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde1c60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fee74035000 tx=0x7fee74047000 comp rx=0 tx=0).stop 2024-04-25T21:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 -- 172.21.15.47:0/4012025959 shutdown_connections 2024-04-25T21:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55af9cdf48d0 0x55af9cdf6ce0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cde4800 0x55af9cde1c60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9f40 0x55af9cde1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 --2- 172.21.15.47:0/4012025959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cdd9620 0x55af9cde4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee5affd640 0 client.15732 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.379+0000 7fee84833740 1 -- 172.21.15.47:0/4012025959 >> 172.21.15.47:0/4012025959 conn(0x55af9cdd5b30 msgr2=0x55af9cddb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.380+0000 7fee84833740 1 -- 172.21.15.47:0/4012025959 shutdown_connections 2024-04-25T21:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.380+0000 7fee84833740 1 -- 172.21.15.47:0/4012025959 wait complete. 2024-04-25T21:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.380+0000 7fee84833740 20 client.15732 trim_cache size 0 max 16384 2024-04-25T21:03:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee84833740 1 Processor -- start 2024-04-25T21:03:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee84833740 1 -- start start 2024-04-25T21:03:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee84833740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 0x55af9cdf02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee84833740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdfc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee84833740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdfcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee84833740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1b25d0 con 0x55af9cdd9f40 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee84833740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1afe40 con 0x55af9cde4800 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee84833740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1adcb0 con 0x55af9cdd9620 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee73fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdfc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee7affd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 0x55af9cdf02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee73fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdfc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50934/0 (socket says 172.21.15.47:50934) 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee73fff640 1 -- 172.21.15.47:0/2920984914 learned_addr learned my addr 172.21.15.47:0/2920984914 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdfcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee73fff640 1 -- 172.21.15.47:0/2920984914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 msgr2=0x55af9cdf02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee73fff640 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 0x55af9cdf02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.395+0000 7fee73fff640 1 -- 172.21.15.47:0/2920984914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 msgr2=0x55af9cdfcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.396+0000 7fee73fff640 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdfcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.396+0000 7fee73fff640 1 -- 172.21.15.47:0/2920984914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9cde6e20 con 0x55af9cdd9f40 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.396+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdfcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.396+0000 7fee73fff640 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdfc5f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fee64019730 tx=0x7fee64005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b9eb9aad4fd7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.396+0000 7fee717fa640 1 -- 172.21.15.47:0/2920984914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6401e040 con 0x55af9cdd9f40 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.396+0000 7fee717fa640 1 -- 172.21.15.47:0/2920984914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6400fa70 con 0x55af9cdd9f40 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.396+0000 7fee717fa640 1 -- 172.21.15.47:0/2920984914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64019890 con 0x55af9cdd9f40 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- 172.21.15.47:0/2920984914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 msgr2=0x55af9cdfc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdfc5f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fee64019730 tx=0x7fee64005d50 comp rx=0 tx=0).stop 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- 172.21.15.47:0/2920984914 shutdown_connections 2024-04-25T21:03:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdfcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdfc5f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 --2- 172.21.15.47:0/2920984914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 0x55af9cdf02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- 172.21.15.47:0/2920984914 >> 172.21.15.47:0/2920984914 conn(0x55af9cdd5b30 msgr2=0x55af9cdd4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- 172.21.15.47:0/2920984914 shutdown_connections 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- 172.21.15.47:0/2920984914 wait complete. 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- start start 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 Processor -- start 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 0x55af9cdea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdeacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1ae300 con 0x55af9cdd9f40 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1af770 con 0x55af9cde4800 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.397+0000 7fee84833740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1ae9d0 con 0x55af9cdd9620 2024-04-25T21:03:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee84833740 10 client.0 _ll_register_callbacks cb 0x7fee74032e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50940/0 (socket says 172.21.15.47:50940) 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 -- 172.21.15.47:0/1628099435 learned_addr learned my addr 172.21.15.47:0/1628099435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee7affd640 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 0x55af9cdea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee7b7fe640 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdeacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 -- 172.21.15.47:0/1628099435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 msgr2=0x55af9cdea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 0x55af9cdea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 -- 172.21.15.47:0/1628099435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 msgr2=0x55af9cdeacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdeacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 -- 172.21.15.47:0/1628099435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9ce59e70 con 0x55af9cdd9f40 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee73fff640 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdea780 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee640197d0 tx=0x7fee640095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2fb9a707ccdf48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee5affd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee5affd640 1 -- 172.21.15.47:0/1628099435 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6401e030 con 0x55af9cdd9f40 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee84833740 1 -- 172.21.15.47:0/1628099435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6400fa70 con 0x55af9cdd9f40 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee5affd640 1 -- 172.21.15.47:0/1628099435 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fee64005070 con 0x55af9cdd9f40 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee5affd640 1 -- 172.21.15.47:0/1628099435 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee64039050 con 0x55af9cdd9f40 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee84833740 10 client.15738 fetch_fsmap learned FSMap version 30 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.398+0000 7fee84833740 1 -- 172.21.15.47:0/1628099435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af9ce5b160 con 0x55af9cdd9f40 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee5affd640 1 -- 172.21.15.47:0/1628099435 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6400fa70 con 0x55af9cdd9f40 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee5affd640 1 -- 172.21.15.47:0/1628099435 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fee64003710 con 0x55af9cdd9f40 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee84833740 10 client.15738 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee84833740 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee84833740 20 client.15738 populate_metadata read hostname 'smithi047' 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee84833740 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55af9ce5b660 0x55af9ce5da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee84833740 4 client.15738 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:03:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee84833740 1 -- 172.21.15.47:0/1628099435 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af9cd46200 con 0x55af9ce5b660 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee7affd640 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55af9ce5b660 0x55af9ce5da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee5affd640 10 client.15738 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.399+0000 7fee7affd640 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55af9ce5b660 0x55af9ce5da80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60ca3c51778f477 server_cookie=19c0025b32b5dd04 in_seq=0 out_seq=0 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee5affd640 1 -- 172.21.15.47:0/1628099435 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af9cd46200 con 0x55af9ce5b660 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee5affd640 10 client.15738 handle_command_reply: tid=0 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 client.15738 shutdown 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 -- 172.21.15.47:0/1628099435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 msgr2=0x55af9cdea780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdea780 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee640197d0 tx=0x7fee640095b0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 -- 172.21.15.47:0/1628099435 shutdown_connections 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55af9ce5b660 0x55af9ce5da80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af9cde4800 0x55af9cdeacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af9cdd9f40 0x55af9cdea780 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 --2- 172.21.15.47:0/1628099435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9cdd9620 0x55af9cdea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 -- 172.21.15.47:0/1628099435 >> 172.21.15.47:0/1628099435 conn(0x55af9cdd5b30 msgr2=0x55af9cdd4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee5affd640 0 client.15738 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 -- 172.21.15.47:0/1628099435 shutdown_connections 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 1 -- 172.21.15.47:0/1628099435 wait complete. 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.408+0000 7fee84833740 20 client.15738 trim_cache size 0 max 16384 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:03:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:03:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 -- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fee680a3dc0 msgr2=0x7fee680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fee680a3dc0 0x7fee680a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee7402f500 tx=0x7fee74032c20 comp rx=0 tx=0).stop 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 -- 172.21.15.47:0/3235082938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee7c1021c0 msgr2=0x7fee7c074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee7c1021c0 0x7fee7c074250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c970 tx=0x7fee6c00ce30 comp rx=0 tx=0).stop 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 -- 172.21.15.47:0/3235082938 shutdown_connections 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee7c074cd0 0x7fee7c1a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee7c107290 0x7fee7c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fee680a3dc0 0x7fee680a6280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 --2- 172.21.15.47:0/3235082938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee7c1021c0 0x7fee7c074250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 -- 172.21.15.47:0/3235082938 >> 172.21.15.47:0/3235082938 conn(0x7fee7c0fda20 msgr2=0x7fee7c0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 -- 172.21.15.47:0/3235082938 shutdown_connections 2024-04-25T21:03:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:03:54.409+0000 7fee81a49640 1 -- 172.21.15.47:0/3235082938 wait complete. 2024-04-25T21:03:54.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:54 smithi186 ceph-mon[32529]: pgmap v120: 97 pgs: 97 active+clean; 88 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 443 op/s 2024-04-25T21:03:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:54 smithi092 ceph-mon[32524]: pgmap v120: 97 pgs: 97 active+clean; 88 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 443 op/s 2024-04-25T21:03:54.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:54 smithi047 ceph-mon[29195]: pgmap v120: 97 pgs: 97 active+clean; 88 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 443 op/s 2024-04-25T21:03:55.728 INFO:tasks.workunit.client.0.smithi047.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-25T21:03:55.729 INFO:tasks.workunit.client.0.smithi047.stdout:All 50000 operations completed A-OK! 2024-04-25T21:03:55.729 INFO:tasks.workunit.client.0.smithi047.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-25T21:03:55.729 INFO:tasks.workunit.client.0.smithi047.stdout:Seed set to 1 2024-04-25T21:03:55.857 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T21:03:55.857 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T21:03:55.859 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T21:03:55.859 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T21:03:55.859 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T21:03:55.862 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T21:03:55.864 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T21:03:55.865 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero length punch hole 2024-04-25T21:03:55.865 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero length punch hole 2024-04-25T21:03:55.865 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero length punch hole 2024-04-25T21:03:55.865 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero size read 2024-04-25T21:03:55.865 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero length punch hole 2024-04-25T21:03:55.865 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero size read 2024-04-25T21:03:55.865 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero length punch hole 2024-04-25T21:03:55.865 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x4310e9 2024-04-25T21:03:55.870 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0x954a45 2024-04-25T21:03:55.898 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x9bc582 2024-04-25T21:03:55.924 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x9eecbc 2024-04-25T21:03:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:56 smithi186 ceph-mon[32529]: pgmap v121: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 638 op/s 2024-04-25T21:03:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:56 smithi092 ceph-mon[32524]: pgmap v121: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 638 op/s 2024-04-25T21:03:56.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:56 smithi047 ceph-mon[29195]: pgmap v121: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 638 op/s 2024-04-25T21:03:57.685 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0x9fdd84 2024-04-25T21:03:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:03:58 smithi186 ceph-mon[32529]: pgmap v122: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 405 op/s 2024-04-25T21:03:58.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:03:58 smithi092 ceph-mon[32524]: pgmap v122: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 405 op/s 2024-04-25T21:03:58.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:03:58 smithi047 ceph-mon[29195]: pgmap v122: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 405 op/s 2024-04-25T21:04:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:00 smithi186 ceph-mon[32529]: pgmap v123: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 405 op/s 2024-04-25T21:04:00.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:00 smithi092 ceph-mon[32524]: pgmap v123: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 405 op/s 2024-04-25T21:04:00.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:00 smithi047 ceph-mon[29195]: pgmap v123: 97 pgs: 97 active+clean; 90 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 405 op/s 2024-04-25T21:04:02.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:02 smithi186 ceph-mon[32529]: pgmap v124: 97 pgs: 97 active+clean; 102 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 762 op/s 2024-04-25T21:04:02.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:02 smithi092 ceph-mon[32524]: pgmap v124: 97 pgs: 97 active+clean; 102 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 762 op/s 2024-04-25T21:04:02.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:02 smithi047 ceph-mon[29195]: pgmap v124: 97 pgs: 97 active+clean; 102 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 762 op/s 2024-04-25T21:04:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:04 smithi186 ceph-mon[32529]: pgmap v125: 97 pgs: 97 active+clean; 102 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 551 op/s 2024-04-25T21:04:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:04 smithi092 ceph-mon[32524]: pgmap v125: 97 pgs: 97 active+clean; 102 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 551 op/s 2024-04-25T21:04:04.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:04 smithi047 ceph-mon[29195]: pgmap v125: 97 pgs: 97 active+clean; 102 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 551 op/s 2024-04-25T21:04:06.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:06 smithi186 ceph-mon[32529]: pgmap v126: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 911 op/s 2024-04-25T21:04:06.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:06 smithi092 ceph-mon[32524]: pgmap v126: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 911 op/s 2024-04-25T21:04:06.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:06 smithi047 ceph-mon[29195]: pgmap v126: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 911 op/s 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 -- 172.21.15.47:0/222740219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 msgr2=0x7efd68102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 --2- 172.21.15.47:0/222740219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 0x7efd68102580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efd64009ca0 tx=0x7efd6402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 -- 172.21.15.47:0/222740219 shutdown_connections 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 --2- 172.21.15.47:0/222740219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd68107270 0x7efd6810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 --2- 172.21.15.47:0/222740219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68102b50 0x7efd68106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 --2- 172.21.15.47:0/222740219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 0x7efd68102580 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 -- 172.21.15.47:0/222740219 >> 172.21.15.47:0/222740219 conn(0x7efd680fda00 msgr2=0x7efd680ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 -- 172.21.15.47:0/222740219 shutdown_connections 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.509+0000 7efd6f320640 1 -- 172.21.15.47:0/222740219 wait complete. 2024-04-25T21:04:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6f320640 1 Processor -- start 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6f320640 1 -- start start 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6f320640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 0x7efd68074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6f320640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68107270 0x7efd68074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6f320640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd68074cd0 0x7efd681a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6f320640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd681121e0 con 0x7efd681021a0 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6f320640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd68112060 con 0x7efd68107270 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6f320640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efd68112360 con 0x7efd68074cd0 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 0x7efd68074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6c8bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68107270 0x7efd68074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 0x7efd68074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60152/0 (socket says 172.21.15.47:60152) 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6d0bc640 1 -- 172.21.15.47:0/1298034315 learned_addr learned my addr 172.21.15.47:0/1298034315 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.510+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd68074cd0 0x7efd681a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6c8bb640 1 -- 172.21.15.47:0/1298034315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd68074cd0 msgr2=0x7efd681a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd68074cd0 0x7efd681a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6c8bb640 1 -- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 msgr2=0x7efd68074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 0x7efd68074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6c8bb640 1 -- 172.21.15.47:0/1298034315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd681a7030 con 0x7efd68107270 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 0x7efd68074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd68074cd0 0x7efd681a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68107270 0x7efd68074790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd5800e9c0 tx=0x7efd5800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd5e7fc640 1 -- 172.21.15.47:0/1298034315 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5800ccc0 con 0x7efd68107270 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd5e7fc640 1 -- 172.21.15.47:0/1298034315 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd580108e0 con 0x7efd68107270 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6f320640 1 -- 172.21.15.47:0/1298034315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd681a7320 con 0x7efd68107270 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd5e7fc640 1 -- 172.21.15.47:0/1298034315 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd580187c0 con 0x7efd68107270 2024-04-25T21:04:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.511+0000 7efd6f320640 1 -- 172.21.15.47:0/1298034315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681a7870 con 0x7efd68107270 2024-04-25T21:04:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.513+0000 7efd5e7fc640 1 -- 172.21.15.47:0/1298034315 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7efd58010430 con 0x7efd68107270 2024-04-25T21:04:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd72033740 1 Processor -- start 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd72033740 1 -- start start 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd72033740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdceb620 0x55b0fdceba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd72033740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd72033740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd72033740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b0fdc50970 con 0x55b0fdcebf40 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd72033740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b0fdbed4a0 con 0x55b0fdcf6800 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd72033740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b0fdc4c930 con 0x55b0fdceb620 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd5e7fc640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7efd540a3df0 0x7efd540a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd6c8bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd6d8bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdceb620 0x55b0fdceba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60164/0 (socket says 172.21.15.47:60164) 2024-04-25T21:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd6d8bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdceb620 0x55b0fdceba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:44496/0 (socket says 172.21.15.47:44496) 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd6d0bc640 1 -- 172.21.15.47:0/2533520961 learned_addr learned my addr 172.21.15.47:0/2533520961 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.514+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7efd540a3df0 0x7efd540a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd5e7fc640 1 -- 172.21.15.47:0/1298034315 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7efd580d0ac0 con 0x7efd68107270 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd6d8bd640 1 -- 172.21.15.47:0/2533520961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcf6800 msgr2=0x55b0fdcf8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/2533520961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd6d8bd640 1 -- 172.21.15.47:0/2533520961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 msgr2=0x55b0fdcf62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/2533520961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd6d8bd640 1 -- 172.21.15.47:0/2533520961 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fd963210 con 0x55b0fdceb620 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/2533520961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdceb620 0x55b0fdceba00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd60005f60 tx=0x7efd6000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd0e306028f1890 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7efd540a3df0 0x7efd540a62b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd60009fd0 tx=0x7efd6001e000 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd537fe640 1 -- 172.21.15.47:0/2533520961 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60011550 con 0x55b0fdceb620 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd537fe640 1 -- 172.21.15.47:0/2533520961 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd600054d0 con 0x55b0fdceb620 2024-04-25T21:04:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.515+0000 7efd537fe640 1 -- 172.21.15.47:0/2533520961 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60004bc0 con 0x55b0fdceb620 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 -- 172.21.15.47:0/2533520961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdceb620 msgr2=0x55b0fdceba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 --2- 172.21.15.47:0/2533520961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdceb620 0x55b0fdceba00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd60005f60 tx=0x7efd6000f790 comp rx=0 tx=0).stop 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 -- 172.21.15.47:0/2533520961 shutdown_connections 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 --2- 172.21.15.47:0/2533520961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 --2- 172.21.15.47:0/2533520961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 --2- 172.21.15.47:0/2533520961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdceb620 0x55b0fdceba00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 -- 172.21.15.47:0/2533520961 >> 172.21.15.47:0/2533520961 conn(0x55b0fdce7b30 msgr2=0x55b0fdce7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 -- 172.21.15.47:0/2533520961 shutdown_connections 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 -- 172.21.15.47:0/2533520961 wait complete. 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 -- start start 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.516+0000 7efd72033740 1 Processor -- start 2024-04-25T21:04:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd72033740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcf61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd72033740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd72033740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd72033740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b0fdcf9370 con 0x55b0fdcf6800 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd72033740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b0fdbeb0b0 con 0x55b0fdceb620 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd72033740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b0fdcf94f0 con 0x55b0fdcebf40 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcf61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcf61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:51708/0 (socket says 172.21.15.47:51708) 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6c8bb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d0bc640 1 -- 172.21.15.47:0/3249281425 learned_addr learned my addr 172.21.15.47:0/3249281425 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd72033740 10 client.0 _ll_register_callbacks cb 0x55b0fdbd8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:04:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d0bc640 1 -- 172.21.15.47:0/3249281425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcebf40 msgr2=0x55b0fdcf34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d0bc640 1 -- 172.21.15.47:0/3249281425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcf6800 msgr2=0x55b0fdcf3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.517+0000 7efd6d0bc640 1 -- 172.21.15.47:0/3249281425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fda2f620 con 0x55b0fdceb620 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcf61c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efd64009430 tx=0x7efd64032f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9818d760ad9a54b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd50ff9640 1 -- 172.21.15.47:0/3249281425 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6404b060 con 0x55b0fdceb620 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd50ff9640 1 -- 172.21.15.47:0/3249281425 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7efd64003b80 con 0x55b0fdceb620 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 1 -- 172.21.15.47:0/3249281425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd600054d0 con 0x55b0fdceb620 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd50ff9640 1 -- 172.21.15.47:0/3249281425 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd640098a0 con 0x55b0fdceb620 2024-04-25T21:04:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 10 client.25373 fetch_fsmap learned FSMap version 30 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 1 -- 172.21.15.47:0/3249281425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0fdd052e0 con 0x55b0fdceb620 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd50ff9640 1 -- 172.21.15.47:0/3249281425 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd600054d0 con 0x55b0fdceb620 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd50ff9640 1 -- 172.21.15.47:0/3249281425 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efd6402f860 con 0x55b0fdceb620 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 10 client.25373 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 20 client.25373 populate_metadata read hostname 'smithi047' 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55b0fdd067a0 0x55b0fdd08bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 4 client.25373 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.518+0000 7efd72033740 1 -- 172.21.15.47:0/3249281425 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0fdc5e390 con 0x55b0fdd067a0 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.519+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55b0fdd067a0 0x55b0fdd08bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.519+0000 7efd50ff9640 10 client.25373 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.519+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55b0fdd067a0 0x55b0fdd08bd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febde7d3508a1513 server_cookie=684f863ed8b413f in_seq=0 out_seq=0 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd50ff9640 1 -- 172.21.15.47:0/3249281425 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0fdc5e390 con 0x55b0fdd067a0 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd50ff9640 10 client.25373 handle_command_reply: tid=0 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 client.25373 shutdown 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 -- 172.21.15.47:0/3249281425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 msgr2=0x55b0fdcf61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcf61c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efd64009430 tx=0x7efd64032f70 comp rx=0 tx=0).stop 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 -- 172.21.15.47:0/3249281425 shutdown_connections 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55b0fdd067a0 0x55b0fdd08bd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcebf40 0x55b0fdcf34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 --2- 172.21.15.47:0/3249281425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcf61c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd50ff9640 0 client.25373 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 -- 172.21.15.47:0/3249281425 >> 172.21.15.47:0/3249281425 conn(0x55b0fdce7b30 msgr2=0x55b0fdced8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 -- 172.21.15.47:0/3249281425 shutdown_connections 2024-04-25T21:04:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 1 -- 172.21.15.47:0/3249281425 wait complete. 2024-04-25T21:04:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.520+0000 7efd72033740 20 client.25373 trim_cache size 0 max 16384 2024-04-25T21:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd72033740 1 Processor -- start 2024-04-25T21:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd72033740 1 -- start start 2024-04-25T21:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd72033740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdceb620 0x55b0fdd02230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd72033740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcebf40 0x55b0fdcee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd72033740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd72033740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd681b25a0 con 0x55b0fdceb620 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd72033740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd681afe10 con 0x55b0fdcebf40 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd72033740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efd681adc80 con 0x55b0fdcf6800 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdceb620 0x55b0fdd02230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdceb620 0x55b0fdd02230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60188/0 (socket says 172.21.15.47:60188) 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d0bc640 1 -- 172.21.15.47:0/1562945361 learned_addr learned my addr 172.21.15.47:0/1562945361 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcebf40 0x55b0fdcee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d0bc640 1 -- 172.21.15.47:0/1562945361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 msgr2=0x55b0fdcf5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d0bc640 1 -- 172.21.15.47:0/1562945361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcebf40 msgr2=0x55b0fdcee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcebf40 0x55b0fdcee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.538+0000 7efd6d0bc640 1 -- 172.21.15.47:0/1562945361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fdcf8bf0 con 0x55b0fdceb620 2024-04-25T21:04:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.539+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdceb620 0x55b0fdd02230 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efd64009bf0 tx=0x7efd6402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b26783de8d3a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.539+0000 7efd52ffd640 1 -- 172.21.15.47:0/1562945361 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64040020 con 0x55b0fdceb620 2024-04-25T21:04:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.539+0000 7efd52ffd640 1 -- 172.21.15.47:0/1562945361 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd64036040 con 0x55b0fdceb620 2024-04-25T21:04:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.539+0000 7efd52ffd640 1 -- 172.21.15.47:0/1562945361 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6404b070 con 0x55b0fdceb620 2024-04-25T21:04:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 -- 172.21.15.47:0/1562945361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdceb620 msgr2=0x55b0fdd02230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdceb620 0x55b0fdd02230 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efd64009bf0 tx=0x7efd6402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 -- 172.21.15.47:0/1562945361 shutdown_connections 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 0x55b0fdcf5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdcebf40 0x55b0fdcee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 --2- 172.21.15.47:0/1562945361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdceb620 0x55b0fdd02230 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 -- 172.21.15.47:0/1562945361 >> 172.21.15.47:0/1562945361 conn(0x55b0fdce7b30 msgr2=0x55b0fdcee260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 -- 172.21.15.47:0/1562945361 shutdown_connections 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 -- 172.21.15.47:0/1562945361 wait complete. 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 -- start start 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 Processor -- start 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcfeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.540+0000 7efd72033740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcfa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd72033740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 0x55b0fdcfa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd72033740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd681ae2d0 con 0x55b0fdcebf40 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd72033740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd681af740 con 0x55b0fdceb620 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd72033740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efd681ae9a0 con 0x55b0fdcf6800 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd72033740 10 client.0 _ll_register_callbacks cb 0x7efd64031930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6c8bb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcfa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6d0bc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcfeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6c8bb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcfa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60198/0 (socket says 172.21.15.47:60198) 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6c8bb640 1 -- 172.21.15.47:0/1203507565 learned_addr learned my addr 172.21.15.47:0/1203507565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6d8bd640 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 0x55b0fdcfa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6d0bc640 1 -- 172.21.15.47:0/1203507565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 msgr2=0x55b0fdcfa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 0x55b0fdcfa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6d0bc640 1 -- 172.21.15.47:0/1203507565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 msgr2=0x55b0fdcfa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcfa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6d0bc640 1 -- 172.21.15.47:0/1203507565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fdd6bd50 con 0x55b0fdceb620 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcfa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd6d0bc640 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcfeca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd64002230 tx=0x7efd64030870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b6b2f43662cea71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd50ff9640 1 -- 172.21.15.47:0/1203507565 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6404b040 con 0x55b0fdceb620 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd72033740 1 -- 172.21.15.47:0/1203507565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd64036040 con 0x55b0fdceb620 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.541+0000 7efd50ff9640 1 -- 172.21.15.47:0/1203507565 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7efd64030e70 con 0x55b0fdceb620 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd50ff9640 1 -- 172.21.15.47:0/1203507565 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd64009540 con 0x55b0fdceb620 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd72033740 10 client.25379 fetch_fsmap learned FSMap version 30 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd72033740 1 -- 172.21.15.47:0/1203507565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0fdd6cf00 con 0x55b0fdceb620 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd50ff9640 1 -- 172.21.15.47:0/1203507565 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd64036040 con 0x55b0fdceb620 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd50ff9640 1 -- 172.21.15.47:0/1203507565 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efd64058430 con 0x55b0fdceb620 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd72033740 10 client.25379 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd72033740 10 client.25379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd72033740 20 client.25379 populate_metadata read hostname 'smithi047' 2024-04-25T21:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd72033740 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55b0fdd6d560 0x55b0fdd6f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd72033740 4 client.25379 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd72033740 1 -- 172.21.15.47:0/1203507565 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0fdc58200 con 0x55b0fdd6d560 2024-04-25T21:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.542+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55b0fdd6d560 0x55b0fdd6f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.543+0000 7efd6c8bb640 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55b0fdd6d560 0x55b0fdd6f950 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4b1722188dd6d3 server_cookie=e25ea85ddf883ddc in_seq=0 out_seq=0 2024-04-25T21:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.543+0000 7efd50ff9640 10 client.25379 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd50ff9640 1 -- 172.21.15.47:0/1203507565 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b0fdc58200 con 0x55b0fdd6d560 2024-04-25T21:04:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd50ff9640 10 client.25379 handle_command_reply: tid=0 2024-04-25T21:04:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 client.25379 shutdown 2024-04-25T21:04:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 -- 172.21.15.47:0/1203507565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 msgr2=0x55b0fdcfeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcfeca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd64002230 tx=0x7efd64030870 comp rx=0 tx=0).stop 2024-04-25T21:04:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 -- 172.21.15.47:0/1203507565 shutdown_connections 2024-04-25T21:04:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55b0fdd6d560 0x55b0fdd6f950 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0fdcf6800 0x55b0fdcfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd50ff9640 0 client.25379 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b0fdcebf40 0x55b0fdcfa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 --2- 172.21.15.47:0/1203507565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b0fdceb620 0x55b0fdcfeca0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 -- 172.21.15.47:0/1203507565 >> 172.21.15.47:0/1203507565 conn(0x55b0fdce7b30 msgr2=0x55b0fdce6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.550+0000 7efd72033740 1 -- 172.21.15.47:0/1203507565 shutdown_connections 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd72033740 1 -- 172.21.15.47:0/1203507565 wait complete. 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd72033740 20 client.25379 trim_cache size 0 max 16384 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:04:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:04:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:04:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 -- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7efd540a3df0 msgr2=0x7efd540a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7efd540a3df0 0x7efd540a62b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd60009fd0 tx=0x7efd6001e000 comp rx=0 tx=0).stop 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 -- 172.21.15.47:0/1298034315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68107270 msgr2=0x7efd68074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68107270 0x7efd68074790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd5800e9c0 tx=0x7efd5800ee80 comp rx=0 tx=0).stop 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 -- 172.21.15.47:0/1298034315 shutdown_connections 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd68074cd0 0x7efd681a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7efd540a3df0 0x7efd540a62b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68107270 0x7efd68074790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 --2- 172.21.15.47:0/1298034315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd681021a0 0x7efd68074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 -- 172.21.15.47:0/1298034315 >> 172.21.15.47:0/1298034315 conn(0x7efd680fda00 msgr2=0x7efd680ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 -- 172.21.15.47:0/1298034315 shutdown_connections 2024-04-25T21:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:07.551+0000 7efd6f320640 1 -- 172.21.15.47:0/1298034315 wait complete. 2024-04-25T21:04:07.978 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0x9ff773 2024-04-25T21:04:08.332 INFO:tasks.workunit.client.0.smithi047.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-25T21:04:08.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:08 smithi047 ceph-mon[29195]: pgmap v127: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 715 op/s 2024-04-25T21:04:08.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:08 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:08.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:08 smithi186 ceph-mon[32529]: pgmap v127: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 715 op/s 2024-04-25T21:04:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:08 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:08 smithi092 ceph-mon[32524]: pgmap v127: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 715 op/s 2024-04-25T21:04:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:08 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:09.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:09 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:09.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:09 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:09.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:09 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:04:09.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:09 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:09.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:09 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:09.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:09 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:04:09.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:09 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:09 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:09.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:09 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:04:10.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:10 smithi186 ceph-mon[32529]: pgmap v128: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 715 op/s 2024-04-25T21:04:10.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:10 smithi092 ceph-mon[32524]: pgmap v128: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 715 op/s 2024-04-25T21:04:10.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:10 smithi047 ceph-mon[29195]: pgmap v128: 97 pgs: 97 active+clean; 109 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 715 op/s 2024-04-25T21:04:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:12 smithi186 ceph-mon[32529]: pgmap v129: 97 pgs: 97 active+clean; 111 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 1.05k op/s 2024-04-25T21:04:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:12 smithi092 ceph-mon[32524]: pgmap v129: 97 pgs: 97 active+clean; 111 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 1.05k op/s 2024-04-25T21:04:12.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:12 smithi047 ceph-mon[29195]: pgmap v129: 97 pgs: 97 active+clean; 111 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 1.05k op/s 2024-04-25T21:04:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:14 smithi186 ceph-mon[32529]: pgmap v130: 97 pgs: 97 active+clean; 111 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.9 MiB/s wr, 690 op/s 2024-04-25T21:04:14.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:14 smithi092 ceph-mon[32524]: pgmap v130: 97 pgs: 97 active+clean; 111 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.9 MiB/s wr, 690 op/s 2024-04-25T21:04:14.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:14 smithi047 ceph-mon[29195]: pgmap v130: 97 pgs: 97 active+clean; 111 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.9 MiB/s wr, 690 op/s 2024-04-25T21:04:15.151 INFO:tasks.workunit.client.0.smithi047.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-25T21:04:16.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:16 smithi186 ceph-mon[32529]: pgmap v131: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 1.02k op/s 2024-04-25T21:04:16.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:16 smithi092 ceph-mon[32524]: pgmap v131: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 1.02k op/s 2024-04-25T21:04:16.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:16 smithi047 ceph-mon[29195]: pgmap v131: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 1.02k op/s 2024-04-25T21:04:18.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:18 smithi186 ceph-mon[32529]: pgmap v132: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 658 op/s 2024-04-25T21:04:18.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:18 smithi092 ceph-mon[32524]: pgmap v132: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 658 op/s 2024-04-25T21:04:18.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:18 smithi047 ceph-mon[29195]: pgmap v132: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 658 op/s 2024-04-25T21:04:18.871 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x9ffb01 2024-04-25T21:04:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 -- 172.21.15.47:0/4086830232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6341031e0 msgr2=0x7fc6341035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 --2- 172.21.15.47:0/4086830232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6341031e0 0x7fc6341035c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009900 tx=0x7fc61c02a510 comp rx=0 tx=0).stop 2024-04-25T21:04:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 -- 172.21.15.47:0/4086830232 shutdown_connections 2024-04-25T21:04:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 --2- 172.21.15.47:0/4086830232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc63410c520 0x7fc63410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 --2- 172.21.15.47:0/4086830232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc634103b00 0x7fc63410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 --2- 172.21.15.47:0/4086830232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6341031e0 0x7fc6341035c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 -- 172.21.15.47:0/4086830232 >> 172.21.15.47:0/4086830232 conn(0x7fc634077fd0 msgr2=0x7fc6340783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 -- 172.21.15.47:0/4086830232 shutdown_connections 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.656+0000 7fc638dfe640 1 -- 172.21.15.47:0/4086830232 wait complete. 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc638dfe640 1 Processor -- start 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc638dfe640 1 -- start start 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc638dfe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341031e0 0x7fc6341986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc638dfe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc634103b00 0x7fc634198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc638dfe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63410c520 0x7fc63419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc638dfe640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341121b0 con 0x7fc634103b00 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc638dfe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc634112030 con 0x7fc63410c520 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc638dfe640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc634112330 con 0x7fc6341031e0 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc631d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc634103b00 0x7fc634198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc632d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63410c520 0x7fc63419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc632d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63410c520 0x7fc63419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:44678/0 (socket says 172.21.15.47:44678) 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc632d9d640 1 -- 172.21.15.47:0/347720165 learned_addr learned my addr 172.21.15.47:0/347720165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.657+0000 7fc63259c640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341031e0 0x7fc6341986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc63259c640 1 -- 172.21.15.47:0/347720165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63410c520 msgr2=0x7fc63419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc63259c640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63410c520 0x7fc63419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc63259c640 1 -- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc634103b00 msgr2=0x7fc634198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc63259c640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc634103b00 0x7fc634198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc63259c640 1 -- 172.21.15.47:0/347720165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63419d6a0 con 0x7fc6341031e0 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc632d9d640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63410c520 0x7fc63419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc631d9b640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc634103b00 0x7fc634198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc63259c640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341031e0 0x7fc6341986b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc61c006010 tx=0x7fc61c004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc62b7fe640 1 -- 172.21.15.47:0/347720165 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c03f070 con 0x7fc6341031e0 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc638dfe640 1 -- 172.21.15.47:0/347720165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63419d930 con 0x7fc6341031e0 2024-04-25T21:04:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc62b7fe640 1 -- 172.21.15.47:0/347720165 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c003ca0 con 0x7fc6341031e0 2024-04-25T21:04:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc62b7fe640 1 -- 172.21.15.47:0/347720165 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c035070 con 0x7fc6341031e0 2024-04-25T21:04:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.658+0000 7fc638dfe640 1 -- 172.21.15.47:0/347720165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341a5220 con 0x7fc6341031e0 2024-04-25T21:04:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc62b7fe640 1 -- 172.21.15.47:0/347720165 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc61c03b460 con 0x7fc6341031e0 2024-04-25T21:04:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc63bbec740 1 Processor -- start 2024-04-25T21:04:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc63bbec740 1 -- start start 2024-04-25T21:04:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950620 0x55bbf8950a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf895b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf895dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc63bbec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bbf88b5970 con 0x55bbf8950620 2024-04-25T21:04:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc63bbec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bbf88524a0 con 0x55bbf8950f40 2024-04-25T21:04:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.661+0000 7fc63bbec740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bbf88b1930 con 0x55bbf895b800 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc62b7fe640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc6200a3dc0 0x7fc6200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf895b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc632d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950620 0x55bbf8950a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc631d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf895dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf895b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:44694/0 (socket says 172.21.15.47:44694) 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 -- 172.21.15.47:0/3147709702 learned_addr learned my addr 172.21.15.47:0/3147709702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc631d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf895dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:34818/0 (socket says 172.21.15.47:34818) 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc632d9d640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc6200a3dc0 0x7fc6200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 -- 172.21.15.47:0/3147709702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 msgr2=0x55bbf895dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 --2- 172.21.15.47:0/3147709702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf895dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 -- 172.21.15.47:0/3147709702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950620 msgr2=0x55bbf8950a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 --2- 172.21.15.47:0/3147709702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950620 0x55bbf8950a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 -- 172.21.15.47:0/3147709702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbf85c8210 con 0x55bbf8950f40 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc631d9b640 1 --2- 172.21.15.47:0/3147709702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf895dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc62b7fe640 1 -- 172.21.15.47:0/347720165 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fc61c0f0c80 con 0x7fc6341031e0 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc632d9d640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc6200a3dc0 0x7fc6200a6280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc62c005c40 tx=0x7fc62c004ab0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.662+0000 7fc63259c640 1 --2- 172.21.15.47:0/3147709702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf895b2c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0c0ac0 tx=0x7fc61c049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625ffb8f0d7a917d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc628ff9640 1 -- 172.21.15.47:0/3147709702 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c048070 con 0x55bbf8950f40 2024-04-25T21:04:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc628ff9640 1 -- 172.21.15.47:0/3147709702 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c0c0f80 con 0x55bbf8950f40 2024-04-25T21:04:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc628ff9640 1 -- 172.21.15.47:0/3147709702 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0bcc80 con 0x55bbf8950f40 2024-04-25T21:04:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 -- 172.21.15.47:0/3147709702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 msgr2=0x55bbf895b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3147709702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf895b2c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0c0ac0 tx=0x7fc61c049000 comp rx=0 tx=0).stop 2024-04-25T21:04:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 -- 172.21.15.47:0/3147709702 shutdown_connections 2024-04-25T21:04:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3147709702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf895dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3147709702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf895b2c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3147709702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950620 0x55bbf8950a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 -- 172.21.15.47:0/3147709702 >> 172.21.15.47:0/3147709702 conn(0x55bbf894cb30 msgr2=0x55bbf894cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 -- 172.21.15.47:0/3147709702 shutdown_connections 2024-04-25T21:04:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.663+0000 7fc63bbec740 1 -- 172.21.15.47:0/3147709702 wait complete. 2024-04-25T21:04:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 1 -- start start 2024-04-25T21:04:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 1 Processor -- start 2024-04-25T21:04:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950620 0x55bbf895b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf8958480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf8958a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bbf895e1f0 con 0x55bbf8950f40 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bbf88500b0 con 0x55bbf8950620 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bbf895e370 con 0x55bbf895b800 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63bbec740 10 client.0 _ll_register_callbacks cb 0x55bbf883dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc631d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf8958480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950620 0x55bbf895b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950620 0x55bbf895b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:44706/0 (socket says 172.21.15.47:44706) 2024-04-25T21:04:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.664+0000 7fc63259c640 1 -- 172.21.15.47:0/525024288 learned_addr learned my addr 172.21.15.47:0/525024288 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63259c640 1 -- 172.21.15.47:0/525024288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 msgr2=0x55bbf8958a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc632d9d640 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf8958a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63259c640 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf8958a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63259c640 1 -- 172.21.15.47:0/525024288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 msgr2=0x55bbf8958480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63259c640 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf8958480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63259c640 1 -- 172.21.15.47:0/525024288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbf8694620 con 0x55bbf8950620 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc631d9b640 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf8958480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc632d9d640 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf8958a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63259c640 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950620 0x55bbf895b190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc61c02aa30 tx=0x7fc61c03b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c29b4553b57a8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc619ffb640 1 -- 172.21.15.47:0/525024288 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c048040 con 0x55bbf8950620 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc619ffb640 1 -- 172.21.15.47:0/525024288 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fc61c0dd0c0 con 0x55bbf8950620 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63bbec740 1 -- 172.21.15.47:0/525024288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61c0c0f80 con 0x55bbf8950620 2024-04-25T21:04:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc619ffb640 1 -- 172.21.15.47:0/525024288 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc61c0bcc80 con 0x55bbf8950620 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63bbec740 10 client.25391 fetch_fsmap learned FSMap version 30 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc63bbec740 1 -- 172.21.15.47:0/525024288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bbf895e1f0 con 0x55bbf8950620 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.665+0000 7fc619ffb640 1 -- 172.21.15.47:0/525024288 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c0c0f80 con 0x55bbf8950620 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc619ffb640 1 -- 172.21.15.47:0/525024288 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc61c04b060 con 0x55bbf8950620 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc63bbec740 10 client.25391 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc63bbec740 10 client.25391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc63bbec740 20 client.25391 populate_metadata read hostname 'smithi047' 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc63bbec740 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bbf896b380 0x55bbf896d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc63bbec740 4 client.25391 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc63bbec740 1 -- 172.21.15.47:0/525024288 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbf88c3390 con 0x55bbf896b380 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc631d9b640 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bbf896b380 0x55bbf896d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc631d9b640 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bbf896b380 0x55bbf896d7a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cca4299d28d744b server_cookie=a530133559bc62f3 in_seq=0 out_seq=0 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.666+0000 7fc619ffb640 10 client.25391 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc619ffb640 1 -- 172.21.15.47:0/525024288 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbf88c3390 con 0x55bbf896b380 2024-04-25T21:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc619ffb640 10 client.25391 handle_command_reply: tid=0 2024-04-25T21:04:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 client.25391 shutdown 2024-04-25T21:04:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 -- 172.21.15.47:0/525024288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950620 msgr2=0x55bbf895b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950620 0x55bbf895b190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc61c02aa30 tx=0x7fc61c03b800 comp rx=0 tx=0).stop 2024-04-25T21:04:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 -- 172.21.15.47:0/525024288 shutdown_connections 2024-04-25T21:04:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bbf896b380 0x55bbf896d7a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf895b800 0x55bbf8958a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc619ffb640 0 client.25391 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf8958480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 --2- 172.21.15.47:0/525024288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950620 0x55bbf895b190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 -- 172.21.15.47:0/525024288 >> 172.21.15.47:0/525024288 conn(0x55bbf894cb30 msgr2=0x55bbf89528b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 -- 172.21.15.47:0/525024288 shutdown_connections 2024-04-25T21:04:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 1 -- 172.21.15.47:0/525024288 wait complete. 2024-04-25T21:04:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.667+0000 7fc63bbec740 20 client.25391 trim_cache size 0 max 16384 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.684+0000 7fc63bbec740 1 Processor -- start 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.684+0000 7fc63bbec740 1 -- start start 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.684+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.684+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf8973070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.684+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf895b800 0x55bbf89735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.684+0000 7fc63bbec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341ae200 con 0x55bbf895b800 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.684+0000 7fc63bbec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341aba70 con 0x55bbf8950f40 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.684+0000 7fc63bbec740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341a98e0 con 0x55bbf8950620 2024-04-25T21:04:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc632d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf895b800 0x55bbf89735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc63259c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc632d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf895b800 0x55bbf89735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60832/0 (socket says 172.21.15.47:60832) 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc632d9d640 1 -- 172.21.15.47:0/3733195663 learned_addr learned my addr 172.21.15.47:0/3733195663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc631d9b640 1 --2- 172.21.15.47:0/3733195663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf8973070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc63259c640 1 -- 172.21.15.47:0/3733195663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 msgr2=0x55bbf8973070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc63259c640 1 --2- 172.21.15.47:0/3733195663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf8973070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc63259c640 1 -- 172.21.15.47:0/3733195663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf895b800 msgr2=0x55bbf89735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc63259c640 1 --2- 172.21.15.47:0/3733195663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf895b800 0x55bbf89735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc63259c640 1 -- 172.21.15.47:0/3733195663 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbf89518e0 con 0x55bbf8950620 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc63259c640 1 --2- 172.21.15.47:0/3733195663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895acc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0b90e0 tx=0x7fc61c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39679a5f1bf23c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc61bfff640 1 -- 172.21.15.47:0/3733195663 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c048040 con 0x55bbf8950620 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc61bfff640 1 -- 172.21.15.47:0/3733195663 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c0c0f80 con 0x55bbf8950620 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.685+0000 7fc61bfff640 1 -- 172.21.15.47:0/3733195663 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c040270 con 0x55bbf8950620 2024-04-25T21:04:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 -- 172.21.15.47:0/3733195663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 msgr2=0x55bbf895acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3733195663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895acc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0b90e0 tx=0x7fc61c000bc0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 -- 172.21.15.47:0/3733195663 shutdown_connections 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3733195663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf895b800 0x55bbf89735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3733195663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf8950f40 0x55bbf8973070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3733195663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895acc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 -- 172.21.15.47:0/3733195663 >> 172.21.15.47:0/3733195663 conn(0x55bbf894cb30 msgr2=0x55bbf894cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 -- 172.21.15.47:0/3733195663 shutdown_connections 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 -- 172.21.15.47:0/3733195663 wait complete. 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.686+0000 7fc63bbec740 1 -- start start 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63bbec740 1 Processor -- start 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf895e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63bbec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf895b800 0x55bbf895ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63bbec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341a9f30 con 0x55bbf8950f40 2024-04-25T21:04:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63bbec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341ab3a0 con 0x55bbf895b800 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63bbec740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341aa600 con 0x55bbf8950620 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63bbec740 10 client.0 _ll_register_callbacks cb 0x7fc61c0bc670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc632d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf895b800 0x55bbf895ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc631d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf895e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc632d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf895b800 0x55bbf895ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:44720/0 (socket says 172.21.15.47:44720) 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc632d9d640 1 -- 172.21.15.47:0/3679440742 learned_addr learned my addr 172.21.15.47:0/3679440742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63259c640 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63259c640 1 -- 172.21.15.47:0/3679440742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf895b800 msgr2=0x55bbf895ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63259c640 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf895b800 0x55bbf895ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63259c640 1 -- 172.21.15.47:0/3679440742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 msgr2=0x55bbf895e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63259c640 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf895e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc63259c640 1 -- 172.21.15.47:0/3679440742 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbf89cfb70 con 0x55bbf8950620 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc632d9d640 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf895b800 0x55bbf895ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.687+0000 7fc631d9b640 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf895e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63259c640 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895e480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0b90e0 tx=0x7fc61c0ee460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b852234552f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc619ffb640 1 -- 172.21.15.47:0/3679440742 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c048040 con 0x55bbf8950620 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc619ffb640 1 -- 172.21.15.47:0/3679440742 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fc61c0b92f0 con 0x55bbf8950620 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 1 -- 172.21.15.47:0/3679440742 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61c0c0f80 con 0x55bbf8950620 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc619ffb640 1 -- 172.21.15.47:0/3679440742 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc61c0bcee0 con 0x55bbf8950620 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 10 client.25387 fetch_fsmap learned FSMap version 30 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 1 -- 172.21.15.47:0/3679440742 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bbf89cfb70 con 0x55bbf8950620 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc619ffb640 1 -- 172.21.15.47:0/3679440742 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c0c0f80 con 0x55bbf8950620 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc619ffb640 1 -- 172.21.15.47:0/3679440742 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc61c0c53b0 con 0x55bbf8950620 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 10 client.25387 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 10 client.25387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 20 client.25387 populate_metadata read hostname 'smithi047' 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bbf89d1240 0x55bbf89d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 4 client.25387 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:04:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.688+0000 7fc63bbec740 1 -- 172.21.15.47:0/3679440742 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbf88bd200 con 0x55bbf89d1240 2024-04-25T21:04:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.689+0000 7fc631d9b640 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bbf89d1240 0x55bbf89d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.689+0000 7fc631d9b640 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bbf89d1240 0x55bbf89d3660 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4641cbfffdcafe server_cookie=ad3aaf839ac9c564 in_seq=0 out_seq=0 2024-04-25T21:04:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.689+0000 7fc619ffb640 10 client.25387 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc619ffb640 1 -- 172.21.15.47:0/3679440742 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbf88bd200 con 0x55bbf89d1240 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc619ffb640 10 client.25387 handle_command_reply: tid=0 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 client.25387 shutdown 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 -- 172.21.15.47:0/3679440742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 msgr2=0x55bbf895e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895e480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0b90e0 tx=0x7fc61c0ee460 comp rx=0 tx=0).stop 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 -- 172.21.15.47:0/3679440742 shutdown_connections 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55bbf89d1240 0x55bbf89d3660 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bbf895b800 0x55bbf895ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc619ffb640 0 client.25387 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbf8950f40 0x55bbf895e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 --2- 172.21.15.47:0/3679440742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bbf8950620 0x55bbf895e480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 -- 172.21.15.47:0/3679440742 >> 172.21.15.47:0/3679440742 conn(0x55bbf894cb30 msgr2=0x55bbf894b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 -- 172.21.15.47:0/3679440742 shutdown_connections 2024-04-25T21:04:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 1 -- 172.21.15.47:0/3679440742 wait complete. 2024-04-25T21:04:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.696+0000 7fc63bbec740 20 client.25387 trim_cache size 0 max 16384 2024-04-25T21:04:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:04:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:04:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:04:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:04:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 -- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc6200a3dc0 msgr2=0x7fc6200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc6200a3dc0 0x7fc6200a6280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc62c005c40 tx=0x7fc62c004ab0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 -- 172.21.15.47:0/347720165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341031e0 msgr2=0x7fc6341986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341031e0 0x7fc6341986b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc61c006010 tx=0x7fc61c004060 comp rx=0 tx=0).stop 2024-04-25T21:04:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 -- 172.21.15.47:0/347720165 shutdown_connections 2024-04-25T21:04:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fc6200a3dc0 0x7fc6200a6280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63410c520 0x7fc63419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc634103b00 0x7fc634198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 --2- 172.21.15.47:0/347720165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341031e0 0x7fc6341986b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 -- 172.21.15.47:0/347720165 >> 172.21.15.47:0/347720165 conn(0x7fc634077fd0 msgr2=0x7fc6340fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 -- 172.21.15.47:0/347720165 shutdown_connections 2024-04-25T21:04:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:20.697+0000 7fc638dfe640 1 -- 172.21.15.47:0/347720165 wait complete. 2024-04-25T21:04:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:20 smithi186 ceph-mon[32529]: pgmap v133: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 658 op/s 2024-04-25T21:04:20.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:20 smithi092 ceph-mon[32524]: pgmap v133: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 658 op/s 2024-04-25T21:04:20.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:20 smithi047 ceph-mon[29195]: pgmap v133: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 658 op/s 2024-04-25T21:04:22.297 INFO:tasks.workunit.client.0.smithi047.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-25T21:04:22.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:22 smithi186 ceph-mon[32529]: pgmap v134: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 971 op/s 2024-04-25T21:04:22.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:22 smithi092 ceph-mon[32524]: pgmap v134: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 971 op/s 2024-04-25T21:04:22.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:22 smithi047 ceph-mon[29195]: pgmap v134: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 971 op/s 2024-04-25T21:04:24.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:24 smithi186 ceph-mon[32529]: pgmap v135: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 640 op/s 2024-04-25T21:04:24.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:24 smithi092 ceph-mon[32524]: pgmap v135: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 640 op/s 2024-04-25T21:04:24.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:24 smithi047 ceph-mon[29195]: pgmap v135: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 640 op/s 2024-04-25T21:04:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:26 smithi186 ceph-mon[32529]: pgmap v136: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 950 op/s 2024-04-25T21:04:26.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:26 smithi092 ceph-mon[32524]: pgmap v136: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 950 op/s 2024-04-25T21:04:26.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:26 smithi047 ceph-mon[29195]: pgmap v136: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 950 op/s 2024-04-25T21:04:28.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:28 smithi186 ceph-mon[32529]: pgmap v137: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T21:04:28.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:28 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:28.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:28 smithi092 ceph-mon[32524]: pgmap v137: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T21:04:28.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:28 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:28.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:28 smithi047 ceph-mon[29195]: pgmap v137: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T21:04:28.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:28 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:30.255 INFO:tasks.workunit.client.0.smithi047.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-25T21:04:30.255 INFO:tasks.workunit.client.0.smithi047.stdout:All 50000 operations completed A-OK! 2024-04-25T21:04:30.255 INFO:tasks.workunit.client.0.smithi047.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-25T21:04:30.256 INFO:tasks.workunit.client.0.smithi047.stdout:Seed set to 1 2024-04-25T21:04:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:30 smithi186 ceph-mon[32529]: pgmap v138: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T21:04:30.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:30 smithi092 ceph-mon[32524]: pgmap v138: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T21:04:30.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:30 smithi047 ceph-mon[29195]: pgmap v138: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T21:04:31.446 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T21:04:31.447 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T21:04:31.450 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T21:04:31.450 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T21:04:31.450 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T21:04:31.452 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T21:04:31.455 INFO:tasks.workunit.client.0.smithi047.stdout:skipping zero size read 2024-04-25T21:04:31.455 INFO:tasks.workunit.client.0.smithi047.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T21:04:31.465 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x1c37b90 2024-04-25T21:04:31.472 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x5bd330d 2024-04-25T21:04:31.488 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0x637bf99 2024-04-25T21:04:31.652 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0x63d9b9c 2024-04-25T21:04:32.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:32 smithi186 ceph-mon[32529]: pgmap v139: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 901 op/s 2024-04-25T21:04:32.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:32 smithi092 ceph-mon[32524]: pgmap v139: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 901 op/s 2024-04-25T21:04:32.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:32 smithi047 ceph-mon[29195]: pgmap v139: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 901 op/s 2024-04-25T21:04:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 -- 172.21.15.47:0/1893781775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 msgr2=0x7f6db41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 --2- 172.21.15.47:0/1893781775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 0x7f6db41014d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009900 tx=0x7f6da402a410 comp rx=0 tx=0).stop 2024-04-25T21:04:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 -- 172.21.15.47:0/1893781775 shutdown_connections 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 --2- 172.21.15.47:0/1893781775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db4106170 0x7f6db410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 --2- 172.21.15.47:0/1893781775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6db4101aa0 0x7f6db4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 --2- 172.21.15.47:0/1893781775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 0x7f6db41014d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 -- 172.21.15.47:0/1893781775 >> 172.21.15.47:0/1893781775 conn(0x7f6db4077fd0 msgr2=0x7f6db40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 -- 172.21.15.47:0/1893781775 shutdown_connections 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.809+0000 7f6db90bb640 1 -- 172.21.15.47:0/1893781775 wait complete. 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.810+0000 7f6db90bb640 1 Processor -- start 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.810+0000 7f6db90bb640 1 -- start start 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db90bb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 0x7f6db4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db90bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db4106170 0x7f6db4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db90bb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6db419d010 0x7f6db419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db90bb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4111120 con 0x7f6db41010f0 2024-04-25T21:04:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db90bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4110fa0 con 0x7f6db4106170 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db90bb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41112a0 con 0x7f6db419d010 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6db419d010 0x7f6db419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 0x7f6db4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 0x7f6db4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56690/0 (socket says 172.21.15.47:56690) 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6db419d010 0x7f6db419d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:54188/0 (socket says 172.21.15.47:54188) 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 -- 172.21.15.47:0/3559946938 learned_addr learned my addr 172.21.15.47:0/3559946938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db259c640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db4106170 0x7f6db4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 -- 172.21.15.47:0/3559946938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6db419d010 msgr2=0x7f6db419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6db419d010 0x7f6db419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 -- 172.21.15.47:0/3559946938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db4106170 msgr2=0x7f6db4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db4106170 0x7f6db4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 -- 172.21.15.47:0/3559946938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db419db00 con 0x7f6db41010f0 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db359e640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6db419d010 0x7f6db419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.811+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 0x7f6db4198750 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005ab0 tx=0x7f6da4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.812+0000 7f6d9bfff640 1 -- 172.21.15.47:0/3559946938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da403f070 con 0x7f6db41010f0 2024-04-25T21:04:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.812+0000 7f6d9bfff640 1 -- 172.21.15.47:0/3559946938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da4031b60 con 0x7f6db41010f0 2024-04-25T21:04:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.812+0000 7f6db90bb640 1 -- 172.21.15.47:0/3559946938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db419dd90 con 0x7f6db41010f0 2024-04-25T21:04:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.812+0000 7f6d9bfff640 1 -- 172.21.15.47:0/3559946938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4035030 con 0x7f6db41010f0 2024-04-25T21:04:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.812+0000 7f6db90bb640 1 -- 172.21.15.47:0/3559946938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db41a5550 con 0x7f6db41010f0 2024-04-25T21:04:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.814+0000 7f6d9bfff640 1 -- 172.21.15.47:0/3559946938 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6da4042070 con 0x7f6db41010f0 2024-04-25T21:04:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6dbbdcb740 1 Processor -- start 2024-04-25T21:04:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6dbbdcb740 1 -- start start 2024-04-25T21:04:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9620 0x560c5a9b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560c5a91e970 con 0x560c5a9b9f40 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560c5a8bb4a0 con 0x560c5a9c4800 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c5a91a930 con 0x560c5a9b9620 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6d9bfff640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6d9c0a3dc0 0x7f6d9c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9620 0x560c5a9b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9620 0x560c5a9b9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:54204/0 (socket says 172.21.15.47:54204) 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db359e640 1 -- 172.21.15.47:0/2482482488 learned_addr learned my addr 172.21.15.47:0/2482482488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db359e640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6d9c0a3dc0 0x7f6d9c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db259c640 1 --2- 172.21.15.47:0/2482482488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6d9bfff640 1 -- 172.21.15.47:0/3559946938 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6da40f0ea0 con 0x7f6db41010f0 2024-04-25T21:04:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db359e640 1 -- 172.21.15.47:0/2482482488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 msgr2=0x560c5a9c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db359e640 1 --2- 172.21.15.47:0/2482482488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db359e640 1 -- 172.21.15.47:0/2482482488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 msgr2=0x560c5a9c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.815+0000 7f6db359e640 1 --2- 172.21.15.47:0/2482482488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.816+0000 7f6db359e640 1 -- 172.21.15.47:0/2482482488 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5a631210 con 0x560c5a9b9620 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.816+0000 7f6db359e640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6d9c0a3dc0 0x7f6d9c0a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099e0 tx=0x7f6dac018000 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.816+0000 7f6db359e640 1 --2- 172.21.15.47:0/2482482488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9620 0x560c5a9b9a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009fd0 tx=0x7f6dac00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adcbd0ac88b556d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.816+0000 7f6d997fa640 1 -- 172.21.15.47:0/2482482488 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac022070 con 0x560c5a9b9620 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.816+0000 7f6d997fa640 1 -- 172.21.15.47:0/2482482488 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac00e4f0 con 0x560c5a9b9620 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.816+0000 7f6d997fa640 1 -- 172.21.15.47:0/2482482488 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac011610 con 0x560c5a9b9620 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2482482488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9620 msgr2=0x560c5a9b9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/2482482488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9620 0x560c5a9b9a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009fd0 tx=0x7f6dac00fa00 comp rx=0 tx=0).stop 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2482482488 shutdown_connections 2024-04-25T21:04:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/2482482488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/2482482488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/2482482488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9620 0x560c5a9b9a00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2482482488 >> 172.21.15.47:0/2482482488 conn(0x560c5a9b5b30 msgr2=0x560c5a9b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2482482488 shutdown_connections 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2482482488 wait complete. 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.817+0000 7f6dbbdcb740 1 -- start start 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6dbbdcb740 1 Processor -- start 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5a9c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9c4800 0x560c5a9c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560c5a9c7370 con 0x560c5a9b9f40 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560c5a8b90b0 con 0x560c5a9b9620 2024-04-25T21:04:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c5a9c74f0 con 0x560c5a9c4800 2024-04-25T21:04:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6dbbdcb740 10 client.0 _ll_register_callbacks cb 0x560c5a8a6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:04:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9c4800 0x560c5a9c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56714/0 (socket says 172.21.15.47:56714) 2024-04-25T21:04:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db259c640 1 -- 172.21.15.47:0/1455042722 learned_addr learned my addr 172.21.15.47:0/1455042722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db359e640 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9c4800 0x560c5a9c1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:54210/0 (socket says 172.21.15.47:54210) 2024-04-25T21:04:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5a9c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db359e640 1 -- 172.21.15.47:0/1455042722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 msgr2=0x560c5a9c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db359e640 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5a9c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db359e640 1 -- 172.21.15.47:0/1455042722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 msgr2=0x560c5a9c1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db359e640 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.818+0000 7f6db359e640 1 -- 172.21.15.47:0/1455042722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c5a6fd620 con 0x560c5a9c4800 2024-04-25T21:04:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6db359e640 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9c4800 0x560c5a9c1a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6dac005c40 tx=0x7f6dac011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af223b22242a3ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6d96ff5640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-25T21:04:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6d96ff5640 1 -- 172.21.15.47:0/1455042722 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac022040 con 0x560c5a9c4800 2024-04-25T21:04:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/1455042722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dac00e4f0 con 0x560c5a9c4800 2024-04-25T21:04:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6d96ff5640 1 -- 172.21.15.47:0/1455042722 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6dac00e6a0 con 0x560c5a9c4800 2024-04-25T21:04:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6d96ff5640 1 -- 172.21.15.47:0/1455042722 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6dac03d050 con 0x560c5a9c4800 2024-04-25T21:04:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6dbbdcb740 10 client.25399 fetch_fsmap learned FSMap version 31 2024-04-25T21:04:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/1455042722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c5a9d32e0 con 0x560c5a9c4800 2024-04-25T21:04:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6d96ff5640 1 -- 172.21.15.47:0/1455042722 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac00e4f0 con 0x560c5a9c4800 2024-04-25T21:04:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6d96ff5640 1 -- 172.21.15.47:0/1455042722 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f6dac0266f0 con 0x560c5a9c4800 2024-04-25T21:04:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6dbbdcb740 10 client.25399 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T21:04:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6dbbdcb740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:04:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.819+0000 7f6dbbdcb740 20 client.25399 populate_metadata read hostname 'smithi047' 2024-04-25T21:04:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.820+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x560c5a9d47a0 0x560c5a9d6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.820+0000 7f6dbbdcb740 4 client.25399 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:04:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.820+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/1455042722 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c5a92c390 con 0x560c5a9d47a0 2024-04-25T21:04:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.820+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x560c5a9d47a0 0x560c5a9d6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.820+0000 7f6d96ff5640 10 client.25399 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.820+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x560c5a9d47a0 0x560c5a9d6b90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18578ddfd713d0b server_cookie=d16eeb4ecc7cd927 in_seq=0 out_seq=0 2024-04-25T21:04:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6d96ff5640 1 -- 172.21.15.47:0/1455042722 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c5a92c390 con 0x560c5a9d47a0 2024-04-25T21:04:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6d96ff5640 10 client.25399 handle_command_reply: tid=0 2024-04-25T21:04:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 client.25399 shutdown 2024-04-25T21:04:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/1455042722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9c4800 msgr2=0x560c5a9c1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9c4800 0x560c5a9c1a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6dac005c40 tx=0x7f6dac011af0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/1455042722 shutdown_connections 2024-04-25T21:04:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x560c5a9d47a0 0x560c5a9d6b90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9c4800 0x560c5a9c1a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9f40 0x560c5a9c1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6d96ff5640 0 client.25399 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/1455042722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5a9c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/1455042722 >> 172.21.15.47:0/1455042722 conn(0x560c5a9b5b30 msgr2=0x560c5a9bb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/1455042722 shutdown_connections 2024-04-25T21:04:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/1455042722 wait complete. 2024-04-25T21:04:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.821+0000 7f6dbbdcb740 20 client.25399 trim_cache size 0 max 16384 2024-04-25T21:04:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6dbbdcb740 1 Processor -- start 2024-04-25T21:04:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6dbbdcb740 1 -- start start 2024-04-25T21:04:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9620 0x560c5a9d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5a9c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41b0370 con 0x560c5a9b9620 2024-04-25T21:04:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41adc10 con 0x560c5a9c4800 2024-04-25T21:04:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41aba80 con 0x560c5a9b9f40 2024-04-25T21:04:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9620 0x560c5a9d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6db259c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5a9c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6db359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9620 0x560c5a9d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56726/0 (socket says 172.21.15.47:56726) 2024-04-25T21:04:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6db259c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5a9c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:54214/0 (socket says 172.21.15.47:54214) 2024-04-25T21:04:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.839+0000 7f6db2d9d640 1 -- 172.21.15.47:0/2594793639 learned_addr learned my addr 172.21.15.47:0/2594793639 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6db359e640 1 -- 172.21.15.47:0/2594793639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 msgr2=0x560c5a9c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6db359e640 1 --2- 172.21.15.47:0/2594793639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5a9c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6db359e640 1 -- 172.21.15.47:0/2594793639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9620 msgr2=0x560c5a9d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6db359e640 1 --2- 172.21.15.47:0/2594793639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9620 0x560c5a9d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6db359e640 1 -- 172.21.15.47:0/2594793639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5a9d71f0 con 0x560c5a9c4800 2024-04-25T21:04:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6db359e640 1 --2- 172.21.15.47:0/2594793639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9dc420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099b0 tx=0x7f6dac00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea596f8c01f2a4fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6d98ff9640 1 -- 172.21.15.47:0/2594793639 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac022070 con 0x560c5a9c4800 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6d98ff9640 1 -- 172.21.15.47:0/2594793639 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac034070 con 0x560c5a9c4800 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.840+0000 7f6d98ff9640 1 -- 172.21.15.47:0/2594793639 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00e4f0 con 0x560c5a9c4800 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2594793639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 msgr2=0x560c5a9dc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/2594793639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9dc420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099b0 tx=0x7f6dac00fa00 comp rx=0 tx=0).stop 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2594793639 shutdown_connections 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/2594793639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9c4800 0x560c5a9dc420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/2594793639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5a9c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/2594793639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9b9620 0x560c5a9d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2594793639 >> 172.21.15.47:0/2594793639 conn(0x560c5a9b5b30 msgr2=0x560c5a9b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2594793639 shutdown_connections 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/2594793639 wait complete. 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.841+0000 7f6dbbdcb740 1 -- start start 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6dbbdcb740 1 Processor -- start 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5aa36b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5aa33080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6dbbdcb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9c4800 0x560c5aa33770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41ac0d0 con 0x560c5a9c4800 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41ad540 con 0x560c5a9b9620 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6dbbdcb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41ac7a0 con 0x560c5a9b9f40 2024-04-25T21:04:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6dbbdcb740 10 client.0 _ll_register_callbacks cb 0x7f6dac00f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:04:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5aa36b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db359e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9c4800 0x560c5aa33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5aa36b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:40716/0 (socket says 172.21.15.47:40716) 2024-04-25T21:04:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db2d9d640 1 -- 172.21.15.47:0/3679049572 learned_addr learned my addr 172.21.15.47:0/3679049572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db259c640 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5aa33080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db2d9d640 1 -- 172.21.15.47:0/3679049572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 msgr2=0x560c5aa33080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5aa33080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db2d9d640 1 -- 172.21.15.47:0/3679049572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9c4800 msgr2=0x560c5aa33770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9c4800 0x560c5aa33770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db2d9d640 1 -- 172.21.15.47:0/3679049572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c5a9cb730 con 0x560c5a9b9620 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.842+0000 7f6db359e640 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9c4800 0x560c5aa33770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6db2d9d640 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5aa36b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6da40bcc00 tx=0x7f6da408c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5ead5e84193013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6d96ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6d96ff5640 1 -- 172.21.15.47:0/3679049572 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4049040 con 0x560c5a9b9620 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6d96ff5640 1 -- 172.21.15.47:0/3679049572 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f6da40bb200 con 0x560c5a9b9620 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/3679049572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dac034070 con 0x560c5a9b9620 2024-04-25T21:04:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6d96ff5640 1 -- 172.21.15.47:0/3679049572 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6da40c5ac0 con 0x560c5a9b9620 2024-04-25T21:04:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 10 client.25415 fetch_fsmap learned FSMap version 31 2024-04-25T21:04:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/3679049572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c5a9ccaa0 con 0x560c5a9b9620 2024-04-25T21:04:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6d96ff5640 1 -- 172.21.15.47:0/3679049572 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac034070 con 0x560c5a9b9620 2024-04-25T21:04:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6d96ff5640 1 -- 172.21.15.47:0/3679049572 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f6da403b460 con 0x560c5a9b9620 2024-04-25T21:04:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 10 client.25415 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T21:04:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:04:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 20 client.25415 populate_metadata read hostname 'smithi047' 2024-04-25T21:04:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x560c5aa43890 0x560c5aa45c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 4 client.25415 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:04:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.843+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/3679049572 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c5a926200 con 0x560c5aa43890 2024-04-25T21:04:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.844+0000 7f6db259c640 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x560c5aa43890 0x560c5aa45c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.844+0000 7f6d96ff5640 10 client.25415 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.844+0000 7f6db259c640 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x560c5aa43890 0x560c5aa45c80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53640abe4ca20e9 server_cookie=a46da213a1a389fd in_seq=0 out_seq=0 2024-04-25T21:04:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.853+0000 7f6d96ff5640 1 -- 172.21.15.47:0/3679049572 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c5a926200 con 0x560c5aa43890 2024-04-25T21:04:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.853+0000 7f6d96ff5640 10 client.25415 handle_command_reply: tid=0 2024-04-25T21:04:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.853+0000 7f6dbbdcb740 1 client.25415 shutdown 2024-04-25T21:04:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/3679049572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 msgr2=0x560c5aa36b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5aa36b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6da40bcc00 tx=0x7f6da408c7e0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/3679049572 shutdown_connections 2024-04-25T21:04:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x560c5aa43890 0x560c5aa45c80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c5a9c4800 0x560c5aa33770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5a9b9f40 0x560c5aa33080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6d96ff5640 0 client.25415 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 --2- 172.21.15.47:0/3679049572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c5a9b9620 0x560c5aa36b10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/3679049572 >> 172.21.15.47:0/3679049572 conn(0x560c5a9b5b30 msgr2=0x560c5a9bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/3679049572 shutdown_connections 2024-04-25T21:04:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 1 -- 172.21.15.47:0/3679049572 wait complete. 2024-04-25T21:04:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.854+0000 7f6dbbdcb740 20 client.25415 trim_cache size 0 max 16384 2024-04-25T21:04:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:04:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:04:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:04:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:04:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 -- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6d9c0a3dc0 msgr2=0x7f6d9c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6d9c0a3dc0 0x7f6d9c0a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099e0 tx=0x7f6dac018000 comp rx=0 tx=0).stop 2024-04-25T21:04:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 -- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 msgr2=0x7f6db4198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 0x7f6db4198750 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005ab0 tx=0x7f6da4003c10 comp rx=0 tx=0).stop 2024-04-25T21:04:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 -- 172.21.15.47:0/3559946938 shutdown_connections 2024-04-25T21:04:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6db419d010 0x7f6db419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f6d9c0a3dc0 0x7f6d9c0a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db4106170 0x7f6db4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 --2- 172.21.15.47:0/3559946938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6db41010f0 0x7f6db4198750 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 -- 172.21.15.47:0/3559946938 >> 172.21.15.47:0/3559946938 conn(0x7f6db4077fd0 msgr2=0x7f6db4104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 -- 172.21.15.47:0/3559946938 shutdown_connections 2024-04-25T21:04:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:33.855+0000 7f6db90bb640 1 -- 172.21.15.47:0/3559946938 wait complete. 2024-04-25T21:04:34.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:34 smithi186 ceph-mon[32529]: pgmap v140: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.5 MiB/s wr, 589 op/s 2024-04-25T21:04:34.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:34 smithi092 ceph-mon[32524]: pgmap v140: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.5 MiB/s wr, 589 op/s 2024-04-25T21:04:34.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:34 smithi047 ceph-mon[29195]: pgmap v140: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.5 MiB/s wr, 589 op/s 2024-04-25T21:04:36.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:36 smithi186 ceph-mon[32529]: pgmap v141: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.5 MiB/s wr, 918 op/s 2024-04-25T21:04:36.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:36 smithi092 ceph-mon[32524]: pgmap v141: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.5 MiB/s wr, 918 op/s 2024-04-25T21:04:36.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:36 smithi047 ceph-mon[29195]: pgmap v141: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.5 MiB/s wr, 918 op/s 2024-04-25T21:04:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:38 smithi186 ceph-mon[32529]: pgmap v142: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.7 MiB/s wr, 608 op/s 2024-04-25T21:04:38.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:38 smithi092 ceph-mon[32524]: pgmap v142: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.7 MiB/s wr, 608 op/s 2024-04-25T21:04:38.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:38 smithi047 ceph-mon[29195]: pgmap v142: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.7 MiB/s wr, 608 op/s 2024-04-25T21:04:40.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:40 smithi186 ceph-mon[32529]: pgmap v143: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.7 MiB/s wr, 608 op/s 2024-04-25T21:04:40.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:40 smithi092 ceph-mon[32524]: pgmap v143: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.7 MiB/s wr, 608 op/s 2024-04-25T21:04:40.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:40 smithi047 ceph-mon[29195]: pgmap v143: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.7 MiB/s wr, 608 op/s 2024-04-25T21:04:41.477 INFO:tasks.workunit.client.0.smithi047.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-25T21:04:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:42 smithi186 ceph-mon[32529]: pgmap v144: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 4.0 MiB/s wr, 1.09k op/s 2024-04-25T21:04:42.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:42 smithi092 ceph-mon[32524]: pgmap v144: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 4.0 MiB/s wr, 1.09k op/s 2024-04-25T21:04:42.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:42 smithi047 ceph-mon[29195]: pgmap v144: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 4.0 MiB/s wr, 1.09k op/s 2024-04-25T21:04:44.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:44 smithi186 ceph-mon[32529]: pgmap v145: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 813 op/s 2024-04-25T21:04:44.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:44 smithi092 ceph-mon[32524]: pgmap v145: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 813 op/s 2024-04-25T21:04:44.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:44 smithi047 ceph-mon[29195]: pgmap v145: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 813 op/s 2024-04-25T21:04:46.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:46 smithi186 ceph-mon[32529]: pgmap v146: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.6 MiB/s wr, 1.30k op/s 2024-04-25T21:04:46.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:46 smithi092 ceph-mon[32524]: pgmap v146: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.6 MiB/s wr, 1.30k op/s 2024-04-25T21:04:46.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:46 smithi047 ceph-mon[29195]: pgmap v146: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.6 MiB/s wr, 1.30k op/s 2024-04-25T21:04:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 -- 172.21.15.47:0/1629597709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad810c5b0 msgr2=0x7f2ad810e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/1629597709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad810c5b0 0x7f2ad810e9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009be0 tx=0x7f2ad002a750 comp rx=0 tx=0).stop 2024-04-25T21:04:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 -- 172.21.15.47:0/1629597709 shutdown_connections 2024-04-25T21:04:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/1629597709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad810c5b0 0x7f2ad810e9e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/1629597709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ad8103b00 0x7f2ad810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/1629597709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ad81031e0 0x7f2ad81035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 -- 172.21.15.47:0/1629597709 >> 172.21.15.47:0/1629597709 conn(0x7f2ad8077f70 msgr2=0x7f2ad8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 -- 172.21.15.47:0/1629597709 shutdown_connections 2024-04-25T21:04:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.968+0000 7f2adf9bb640 1 -- 172.21.15.47:0/1629597709 wait complete. 2024-04-25T21:04:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adf9bb640 1 Processor -- start 2024-04-25T21:04:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adf9bb640 1 -- start start 2024-04-25T21:04:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adf9bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad81031e0 0x7f2ad81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adf9bb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ad8103b00 0x7f2ad8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adf9bb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ad810c5b0 0x7f2ad819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adf9bb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81121d0 con 0x7f2ad810c5b0 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adf9bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8112050 con 0x7f2ad81031e0 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adf9bb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8112350 con 0x7f2ad8103b00 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2addf58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ad810c5b0 0x7f2ad819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2add757640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad81031e0 0x7f2ad81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2adcf56640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ad8103b00 0x7f2ad8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2add757640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad81031e0 0x7f2ad81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:39532/0 (socket says 172.21.15.47:39532) 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.969+0000 7f2add757640 1 -- 172.21.15.47:0/957825712 learned_addr learned my addr 172.21.15.47:0/957825712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2addf58640 1 -- 172.21.15.47:0/957825712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ad8103b00 msgr2=0x7f2ad8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2addf58640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ad8103b00 0x7f2ad8198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2addf58640 1 -- 172.21.15.47:0/957825712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad81031e0 msgr2=0x7f2ad81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2addf58640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad81031e0 0x7f2ad81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2addf58640 1 -- 172.21.15.47:0/957825712 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad819d670 con 0x7f2ad810c5b0 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2adcf56640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ad8103b00 0x7f2ad8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2addf58640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ad810c5b0 0x7f2ad819cf40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0034040 tx=0x7f2ad0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2ace7fc640 1 -- 172.21.15.47:0/957825712 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0041020 con 0x7f2ad810c5b0 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2adf9bb640 1 -- 172.21.15.47:0/957825712 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad819d900 con 0x7f2ad810c5b0 2024-04-25T21:04:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2ace7fc640 1 -- 172.21.15.47:0/957825712 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad0003e10 con 0x7f2ad810c5b0 2024-04-25T21:04:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2ace7fc640 1 -- 172.21.15.47:0/957825712 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0036040 con 0x7f2ad810c5b0 2024-04-25T21:04:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.970+0000 7f2adf9bb640 1 -- 172.21.15.47:0/957825712 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad81a51f0 con 0x7f2ad810c5b0 2024-04-25T21:04:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.977+0000 7f2ae27ab740 1 Processor -- start 2024-04-25T21:04:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2ace7fc640 1 -- 172.21.15.47:0/957825712 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f2ad0044020 con 0x7f2ad810c5b0 2024-04-25T21:04:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2ae27ab740 1 -- start start 2024-04-25T21:04:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcaea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcbbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56394bc13970 con 0x56394bcb9800 2024-04-25T21:04:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56394bbb04a0 con 0x56394bcae620 2024-04-25T21:04:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56394bc0f930 con 0x56394bcaef40 2024-04-25T21:04:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2add757640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2adcf56640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcaea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2add757640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:53652/0 (socket says 172.21.15.47:53652) 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.978+0000 7f2add757640 1 -- 172.21.15.47:0/864341959 learned_addr learned my addr 172.21.15.47:0/864341959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2adcf56640 1 --2- 172.21.15.47:0/864341959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcbbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2add757640 1 -- 172.21.15.47:0/864341959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 msgr2=0x56394bcaea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2add757640 1 --2- 172.21.15.47:0/864341959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcaea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2add757640 1 -- 172.21.15.47:0/864341959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 msgr2=0x56394bcbbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2add757640 1 --2- 172.21.15.47:0/864341959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcbbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2add757640 1 -- 172.21.15.47:0/864341959 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56394b926210 con 0x56394bcaef40 2024-04-25T21:04:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2add757640 1 --2- 172.21.15.47:0/864341959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb92c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009d80 tx=0x7f2ad4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c505624734dc181b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2ace7fc640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ac40a3dc0 0x7f2ac40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2ac37fe640 1 -- 172.21.15.47:0/864341959 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad401e070 con 0x56394bcaef40 2024-04-25T21:04:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2adcf56640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ac40a3dc0 0x7f2ac40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2ac37fe640 1 -- 172.21.15.47:0/864341959 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad40035f0 con 0x56394bcaef40 2024-04-25T21:04:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2ac37fe640 1 -- 172.21.15.47:0/864341959 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40196b0 con 0x56394bcaef40 2024-04-25T21:04:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.979+0000 7f2ace7fc640 1 -- 172.21.15.47:0/957825712 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f2ad00f1290 con 0x7f2ad810c5b0 2024-04-25T21:04:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 -- 172.21.15.47:0/864341959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 msgr2=0x56394bcb92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/864341959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb92c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009d80 tx=0x7f2ad4011810 comp rx=0 tx=0).stop 2024-04-25T21:04:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2adcf56640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ac40a3dc0 0x7f2ac40a6280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x56394bcabf50 tx=0x7f2ac8005f90 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 -- 172.21.15.47:0/864341959 shutdown_connections 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/864341959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcbbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/864341959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb92c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/864341959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcaea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 -- 172.21.15.47:0/864341959 >> 172.21.15.47:0/864341959 conn(0x56394bcaab30 msgr2=0x56394bcaaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 -- 172.21.15.47:0/864341959 shutdown_connections 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.980+0000 7f2ae27ab740 1 -- 172.21.15.47:0/864341959 wait complete. 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.981+0000 7f2ae27ab740 1 -- start start 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.981+0000 7f2ae27ab740 1 Processor -- start 2024-04-25T21:04:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.981+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcae620 0x56394bcb9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.981+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcaef40 0x56394bcb6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.981+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcb9800 0x56394bcb6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.981+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56394bcc0840 con 0x56394bcaef40 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.981+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56394bbae0b0 con 0x56394bcb9800 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.981+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56394bcc09c0 con 0x56394bcae620 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2ae27ab740 10 client.0 _ll_register_callbacks cb 0x56394bb9bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2adcf56640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcaef40 0x56394bcb6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2addf58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcb9800 0x56394bcb6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2addf58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcb9800 0x56394bcb6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:39558/0 (socket says 172.21.15.47:39558) 2024-04-25T21:04:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2add757640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcae620 0x56394bcb9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2addf58640 1 -- 172.21.15.47:0/3633944966 learned_addr learned my addr 172.21.15.47:0/3633944966 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2addf58640 1 -- 172.21.15.47:0/3633944966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcae620 msgr2=0x56394bcb9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2addf58640 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcae620 0x56394bcb9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2addf58640 1 -- 172.21.15.47:0/3633944966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcaef40 msgr2=0x56394bcb6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2addf58640 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcaef40 0x56394bcb6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.982+0000 7f2addf58640 1 -- 172.21.15.47:0/3633944966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56394b9f2620 con 0x56394bcb9800 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.983+0000 7f2ac0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.983+0000 7f2addf58640 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcb9800 0x56394bcb6960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00c5980 tx=0x7f2ad00bd0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae2aa025a8483b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.983+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/3633944966 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00bd240 con 0x56394bcb9800 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.983+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/3633944966 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f2ad00c1160 con 0x56394bcb9800 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.983+0000 7f2ae27ab740 1 -- 172.21.15.47:0/3633944966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad40035f0 con 0x56394bcb9800 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.983+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/3633944966 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ad00c15f0 con 0x56394bcb9800 2024-04-25T21:04:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.983+0000 7f2ae27ab740 10 client.25427 fetch_fsmap learned FSMap version 31 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.983+0000 7f2ae27ab740 1 -- 172.21.15.47:0/3633944966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56394bccc790 con 0x56394bcb9800 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/3633944966 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad40035f0 con 0x56394bcb9800 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/3633944966 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f2ad00d1ce0 con 0x56394bcb9800 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2ae27ab740 10 client.25427 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2ae27ab740 10 client.25427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2ae27ab740 20 client.25427 populate_metadata read hostname 'smithi047' 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56394bccdce0 0x56394bcd0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2ae27ab740 4 client.25427 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2ae27ab740 1 -- 172.21.15.47:0/3633944966 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56394bc21390 con 0x56394bccdce0 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.984+0000 7f2add757640 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56394bccdce0 0x56394bcd0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.985+0000 7f2add757640 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56394bccdce0 0x56394bcd0160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcff37bdfb4fca19 server_cookie=f4e0dc3e59b772e3 in_seq=0 out_seq=0 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.985+0000 7f2ac0ff9640 10 client.25427 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.985+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/3633944966 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56394bc21390 con 0x56394bccdce0 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.985+0000 7f2ac0ff9640 10 client.25427 handle_command_reply: tid=0 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.985+0000 7f2ae27ab740 1 client.25427 shutdown 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 -- 172.21.15.47:0/3633944966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcb9800 msgr2=0x56394bcb6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcb9800 0x56394bcb6960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00c5980 tx=0x7f2ad00bd0d0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 -- 172.21.15.47:0/3633944966 shutdown_connections 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56394bccdce0 0x56394bcd0160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcb9800 0x56394bcb6960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcaef40 0x56394bcb6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/3633944966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcae620 0x56394bcb9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ac0ff9640 0 client.25427 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 -- 172.21.15.47:0/3633944966 >> 172.21.15.47:0/3633944966 conn(0x56394bcaab30 msgr2=0x56394bcb0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 -- 172.21.15.47:0/3633944966 shutdown_connections 2024-04-25T21:04:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 1 -- 172.21.15.47:0/3633944966 wait complete. 2024-04-25T21:04:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:46.986+0000 7f2ae27ab740 20 client.25427 trim_cache size 0 max 16384 2024-04-25T21:04:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2ae27ab740 1 Processor -- start 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2ae27ab740 1 -- start start 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcb8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81ae100 con 0x56394bcb9800 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81ab9a0 con 0x56394bcae620 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81a9810 con 0x56394bcaef40 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2addf58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcb8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2adcf56640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2addf58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcb8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44224/0 (socket says 172.21.15.47:44224) 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2adcf56640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:53672/0 (socket says 172.21.15.47:53672) 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2add757640 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2addf58640 1 -- 172.21.15.47:0/1731269539 learned_addr learned my addr 172.21.15.47:0/1731269539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2addf58640 1 -- 172.21.15.47:0/1731269539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 msgr2=0x56394bcb06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2addf58640 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2addf58640 1 -- 172.21.15.47:0/1731269539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 msgr2=0x56394bcc9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2addf58640 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2addf58640 1 -- 172.21.15.47:0/1731269539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56394bcaed70 con 0x56394bcb9800 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.002+0000 7f2add757640 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.003+0000 7f2addf58640 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcb8c60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00eeff0 tx=0x7f2ad0030e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55339fd80bb91a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.003+0000 7f2ac2ffd640 1 -- 172.21.15.47:0/1731269539 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00bc990 con 0x56394bcb9800 2024-04-25T21:04:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.003+0000 7f2ac2ffd640 1 -- 172.21.15.47:0/1731269539 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad00bd240 con 0x56394bcb9800 2024-04-25T21:04:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.003+0000 7f2ac2ffd640 1 -- 172.21.15.47:0/1731269539 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00c13c0 con 0x56394bcb9800 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 -- 172.21.15.47:0/1731269539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 msgr2=0x56394bcb8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcb8c60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00eeff0 tx=0x7f2ad0030e30 comp rx=0 tx=0).stop 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 -- 172.21.15.47:0/1731269539 shutdown_connections 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bcb8c60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcb06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/1731269539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 -- 172.21.15.47:0/1731269539 >> 172.21.15.47:0/1731269539 conn(0x56394bcaab30 msgr2=0x56394bcaffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 -- 172.21.15.47:0/1731269539 shutdown_connections 2024-04-25T21:04:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 -- 172.21.15.47:0/1731269539 wait complete. 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 -- start start 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.004+0000 7f2ae27ab740 1 Processor -- start 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcc1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2ae27ab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bd322d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81a9e60 con 0x56394bcb9800 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81ab2d0 con 0x56394bcae620 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2ae27ab740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81aa530 con 0x56394bcaef40 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2ae27ab740 10 client.0 _ll_register_callbacks cb 0x7f2ad00b94e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2addf58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bd322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2addf58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bd322d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44228/0 (socket says 172.21.15.47:44228) 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2addf58640 1 -- 172.21.15.47:0/620428880 learned_addr learned my addr 172.21.15.47:0/620428880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2adcf56640 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcc1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2add757640 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2addf58640 1 -- 172.21.15.47:0/620428880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 msgr2=0x56394bcc1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2addf58640 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcc1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2addf58640 1 -- 172.21.15.47:0/620428880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 msgr2=0x56394bcc0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2addf58640 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.005+0000 7f2addf58640 1 -- 172.21.15.47:0/620428880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56394bd32810 con 0x56394bcb9800 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2addf58640 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bd322d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00bd1d0 tx=0x7f2ad00bd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71747bf6e71cf705 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ac0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/620428880 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00a0020 con 0x56394bcb9800 2024-04-25T21:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/620428880 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f2ad00a64f0 con 0x56394bcb9800 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 1 -- 172.21.15.47:0/620428880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad00bd240 con 0x56394bcb9800 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/620428880 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ad00bc990 con 0x56394bcb9800 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 10 client.15810 fetch_fsmap learned FSMap version 31 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 1 -- 172.21.15.47:0/620428880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56394bd32810 con 0x56394bcb9800 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/620428880 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad00bd240 con 0x56394bcb9800 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/620428880 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f2ad00d1ce0 con 0x56394bcb9800 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 10 client.15810 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 20 client.15810 populate_metadata read hostname 'smithi047' 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56394bd33db0 0x56394bd361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 4 client.15810 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.006+0000 7f2ae27ab740 1 -- 172.21.15.47:0/620428880 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56394bc1b200 con 0x56394bd33db0 2024-04-25T21:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.007+0000 7f2add757640 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56394bd33db0 0x56394bd361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.007+0000 7f2ac0ff9640 10 client.15810 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.007+0000 7f2add757640 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56394bd33db0 0x56394bd361d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd5483b379e6a8a server_cookie=131f08b2e86c7202 in_seq=0 out_seq=0 2024-04-25T21:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ac0ff9640 1 -- 172.21.15.47:0/620428880 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56394bc1b200 con 0x56394bd33db0 2024-04-25T21:04:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ac0ff9640 10 client.15810 handle_command_reply: tid=0 2024-04-25T21:04:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 client.15810 shutdown 2024-04-25T21:04:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 -- 172.21.15.47:0/620428880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 msgr2=0x56394bd322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bd322d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00bd1d0 tx=0x7f2ad00bd0d0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 -- 172.21.15.47:0/620428880 shutdown_connections 2024-04-25T21:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x56394bd33db0 0x56394bd361d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ac0ff9640 0 client.15810 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56394bcb9800 0x56394bd322d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56394bcaef40 0x56394bcc1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 --2- 172.21.15.47:0/620428880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56394bcae620 0x56394bcc0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.014+0000 7f2ae27ab740 1 -- 172.21.15.47:0/620428880 >> 172.21.15.47:0/620428880 conn(0x56394bcaab30 msgr2=0x56394bcb0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.018+0000 7f2ae27ab740 1 -- 172.21.15.47:0/620428880 shutdown_connections 2024-04-25T21:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.018+0000 7f2ae27ab740 1 -- 172.21.15.47:0/620428880 wait complete. 2024-04-25T21:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.018+0000 7f2ae27ab740 20 client.15810 trim_cache size 0 max 16384 2024-04-25T21:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:04:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:04:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 -- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ac40a3dc0 msgr2=0x7f2ac40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ac40a3dc0 0x7f2ac40a6280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x56394bcabf50 tx=0x7f2ac8005f90 comp rx=0 tx=0).stop 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 -- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ad810c5b0 msgr2=0x7f2ad819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ad810c5b0 0x7f2ad819cf40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0034040 tx=0x7f2ad0003c20 comp rx=0 tx=0).stop 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 -- 172.21.15.47:0/957825712 shutdown_connections 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ad810c5b0 0x7f2ad819cf40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ad8103b00 0x7f2ad8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2ac40a3dc0 0x7f2ac40a6280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 --2- 172.21.15.47:0/957825712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad81031e0 0x7f2ad81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.019+0000 7f2adf9bb640 1 -- 172.21.15.47:0/957825712 >> 172.21.15.47:0/957825712 conn(0x7f2ad8077f70 msgr2=0x7f2ad80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.020+0000 7f2adf9bb640 1 -- 172.21.15.47:0/957825712 shutdown_connections 2024-04-25T21:04:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:04:47.020+0000 7f2adf9bb640 1 -- 172.21.15.47:0/957825712 wait complete. 2024-04-25T21:04:47.334 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x63e5d2a 2024-04-25T21:04:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:48 smithi186 ceph-mon[32529]: pgmap v147: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 969 op/s 2024-04-25T21:04:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:48 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:48 smithi092 ceph-mon[32524]: pgmap v147: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 969 op/s 2024-04-25T21:04:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:48 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:48.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:48 smithi047 ceph-mon[29195]: pgmap v147: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 969 op/s 2024-04-25T21:04:48.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:48 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:49.637 INFO:tasks.workunit.client.0.smithi047.stdout:truncating to largest ever: 0x63e9b52 2024-04-25T21:04:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:50 smithi186 ceph-mon[32529]: pgmap v148: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 969 op/s 2024-04-25T21:04:50.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:50 smithi092 ceph-mon[32524]: pgmap v148: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 969 op/s 2024-04-25T21:04:50.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:50 smithi047 ceph-mon[29195]: pgmap v148: 97 pgs: 97 active+clean; 168 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 969 op/s 2024-04-25T21:04:51.425 INFO:tasks.workunit.client.0.smithi047.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-25T21:04:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:52 smithi186 ceph-mon[32529]: pgmap v149: 97 pgs: 97 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 3.8 MiB/s wr, 1.43k op/s 2024-04-25T21:04:52.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:52 smithi092 ceph-mon[32524]: pgmap v149: 97 pgs: 97 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 3.8 MiB/s wr, 1.43k op/s 2024-04-25T21:04:52.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:52 smithi047 ceph-mon[29195]: pgmap v149: 97 pgs: 97 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 3.8 MiB/s wr, 1.43k op/s 2024-04-25T21:04:53.124 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0x63ed6b9 2024-04-25T21:04:53.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:53 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:53.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:53 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:53.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:53 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:04:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:54 smithi186 ceph-mon[32529]: pgmap v150: 97 pgs: 97 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 948 op/s 2024-04-25T21:04:54.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:54 smithi092 ceph-mon[32524]: pgmap v150: 97 pgs: 97 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 948 op/s 2024-04-25T21:04:54.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:54 smithi047 ceph-mon[29195]: pgmap v150: 97 pgs: 97 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.6 MiB/s wr, 948 op/s 2024-04-25T21:04:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:56 smithi186 ceph-mon[32529]: pgmap v151: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.9 MiB/s wr, 1.41k op/s 2024-04-25T21:04:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:56 smithi092 ceph-mon[32524]: pgmap v151: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.9 MiB/s wr, 1.41k op/s 2024-04-25T21:04:56.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:56 smithi047 ceph-mon[29195]: pgmap v151: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.9 MiB/s wr, 1.41k op/s 2024-04-25T21:04:59.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:04:58 smithi092 ceph-mon[32524]: pgmap v152: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 920 op/s 2024-04-25T21:04:59.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:04:58 smithi047 ceph-mon[29195]: pgmap v152: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 920 op/s 2024-04-25T21:04:59.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:04:58 smithi186 ceph-mon[32529]: pgmap v152: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 920 op/s 2024-04-25T21:05:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 -- 172.21.15.47:0/2535082256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c1010f0 msgr2=0x7fe80c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 --2- 172.21.15.47:0/2535082256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c1010f0 0x7fe80c1014d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe804009ca0 tx=0x7fe80402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 -- 172.21.15.47:0/2535082256 shutdown_connections 2024-04-25T21:05:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 --2- 172.21.15.47:0/2535082256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe80c106170 0x7fe80c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 --2- 172.21.15.47:0/2535082256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80c101aa0 0x7fe80c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 --2- 172.21.15.47:0/2535082256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c1010f0 0x7fe80c1014d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 -- 172.21.15.47:0/2535082256 >> 172.21.15.47:0/2535082256 conn(0x7fe80c077fd0 msgr2=0x7fe80c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 -- 172.21.15.47:0/2535082256 shutdown_connections 2024-04-25T21:05:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.137+0000 7fe8109ad640 1 -- 172.21.15.47:0/2535082256 wait complete. 2024-04-25T21:05:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe8109ad640 1 Processor -- start 2024-04-25T21:05:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe8109ad640 1 -- start start 2024-04-25T21:05:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe8109ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80c1010f0 0x7fe80c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe8109ad640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe80c106170 0x7fe80c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe8109ad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c199dd0 0x7fe80c1a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe8109ad640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1110e0 con 0x7fe80c199dd0 2024-04-25T21:05:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe8109ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c110f60 con 0x7fe80c1010f0 2024-04-25T21:05:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe8109ad640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c111260 con 0x7fe80c106170 2024-04-25T21:05:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c199dd0 0x7fe80c1a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80c1010f0 0x7fe80c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c199dd0 0x7fe80c1a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44676/0 (socket says 172.21.15.47:44676) 2024-04-25T21:05:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe80ad9d640 1 -- 172.21.15.47:0/719895600 learned_addr learned my addr 172.21.15.47:0/719895600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.138+0000 7fe809d9b640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe80c106170 0x7fe80c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe80a59c640 1 -- 172.21.15.47:0/719895600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe80c106170 msgr2=0x7fe80c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe80a59c640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe80c106170 0x7fe80c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe80a59c640 1 -- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c199dd0 msgr2=0x7fe80c1a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe80a59c640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c199dd0 0x7fe80c1a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe80a59c640 1 -- 172.21.15.47:0/719895600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c1a4b50 con 0x7fe80c1010f0 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c199dd0 0x7fe80c1a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe80a59c640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80c1010f0 0x7fe80c199300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe804009bf0 tx=0x7fe804009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe8037fe640 1 -- 172.21.15.47:0/719895600 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804003cb0 con 0x7fe80c1010f0 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe8109ad640 1 -- 172.21.15.47:0/719895600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c1a4de0 con 0x7fe80c1010f0 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe8037fe640 1 -- 172.21.15.47:0/719895600 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe804003e50 con 0x7fe80c1010f0 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe8037fe640 1 -- 172.21.15.47:0/719895600 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80402fc90 con 0x7fe80c1010f0 2024-04-25T21:05:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.139+0000 7fe8109ad640 1 -- 172.21.15.47:0/719895600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c1a52a0 con 0x7fe80c1010f0 2024-04-25T21:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.142+0000 7fe8137a6740 1 Processor -- start 2024-04-25T21:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.142+0000 7fe8137a6740 1 -- start start 2024-04-25T21:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda516a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516f40 0x564fda5212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda523bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe8137a6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564fda47b970 con 0x564fda521800 2024-04-25T21:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe8137a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564fda4184a0 con 0x564fda516f40 2024-04-25T21:05:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe8137a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564fda477930 con 0x564fda516620 2024-04-25T21:05:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda516a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda516a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:38484/0 (socket says 172.21.15.47:38484) 2024-04-25T21:05:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe80ad9d640 1 -- 172.21.15.47:0/1377300497 learned_addr learned my addr 172.21.15.47:0/1377300497 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe80ad9d640 1 -- 172.21.15.47:0/1377300497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516f40 msgr2=0x564fda5212c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T21:05:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/1377300497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516f40 0x564fda5212c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe80ad9d640 1 -- 172.21.15.47:0/1377300497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 msgr2=0x564fda523bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/1377300497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda523bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.143+0000 7fe80ad9d640 1 -- 172.21.15.47:0/1377300497 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fda18e210 con 0x564fda516620 2024-04-25T21:05:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.144+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/1377300497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda516a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009fc0 tx=0x7fe7fc011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4762994196ee2141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.144+0000 7fe800ff9640 1 -- 172.21.15.47:0/1377300497 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc023070 con 0x564fda516620 2024-04-25T21:05:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.145+0000 7fe8037fe640 1 -- 172.21.15.47:0/719895600 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe804004710 con 0x7fe80c1010f0 2024-04-25T21:05:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.146+0000 7fe8037fe640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fe7f80a3e40 0x7fe7f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.146+0000 7fe800ff9640 1 -- 172.21.15.47:0/1377300497 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7fc00fcf0 con 0x564fda516620 2024-04-25T21:05:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.146+0000 7fe800ff9640 1 -- 172.21.15.47:0/1377300497 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc016dc0 con 0x564fda516620 2024-04-25T21:05:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.146+0000 7fe8037fe640 1 -- 172.21.15.47:0/719895600 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fe8040f1d00 con 0x7fe80c1010f0 2024-04-25T21:05:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.147+0000 7fe8137a6740 1 -- 172.21.15.47:0/1377300497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 msgr2=0x564fda516a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.148+0000 7fe8137a6740 1 --2- 172.21.15.47:0/1377300497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda516a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009fc0 tx=0x7fe7fc011a00 comp rx=0 tx=0).stop 2024-04-25T21:05:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.148+0000 7fe8137a6740 1 -- 172.21.15.47:0/1377300497 shutdown_connections 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.148+0000 7fe8137a6740 1 --2- 172.21.15.47:0/1377300497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda523bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.148+0000 7fe8137a6740 1 --2- 172.21.15.47:0/1377300497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516f40 0x564fda5212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.148+0000 7fe8137a6740 1 --2- 172.21.15.47:0/1377300497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda516a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.148+0000 7fe8137a6740 1 -- 172.21.15.47:0/1377300497 >> 172.21.15.47:0/1377300497 conn(0x564fda512b30 msgr2=0x564fda512f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.148+0000 7fe80a59c640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fe7f80a3e40 0x7fe7f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.149+0000 7fe8137a6740 1 -- 172.21.15.47:0/1377300497 shutdown_connections 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.149+0000 7fe8137a6740 1 -- 172.21.15.47:0/1377300497 wait complete. 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.149+0000 7fe80a59c640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fe7f80a3e40 0x7fe7f80a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x564fda517bc0 tx=0x7fe80402a8c0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.149+0000 7fe8137a6740 1 -- start start 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe8137a6740 1 Processor -- start 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516620 0x564fda521150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516f40 0x564fda51e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda51e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe8137a6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564fda524430 con 0x564fda516f40 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe8137a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564fda4160b0 con 0x564fda516620 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe8137a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564fda5245b0 con 0x564fda521800 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe8137a6740 10 client.0 _ll_register_callbacks cb 0x564fda403d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe809d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516f40 0x564fda51e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda51e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe809d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516f40 0x564fda51e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44702/0 (socket says 172.21.15.47:44702) 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda51e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:38496/0 (socket says 172.21.15.47:38496) 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe809d9b640 1 -- 172.21.15.47:0/3137440462 learned_addr learned my addr 172.21.15.47:0/3137440462 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516620 0x564fda521150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe809d9b640 1 -- 172.21.15.47:0/3137440462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 msgr2=0x564fda51e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe809d9b640 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda51e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe809d9b640 1 -- 172.21.15.47:0/3137440462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516620 msgr2=0x564fda521150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe809d9b640 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516620 0x564fda521150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe809d9b640 1 -- 172.21.15.47:0/3137440462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fda25a620 con 0x564fda516f40 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.150+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda51e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe809d9b640 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516f40 0x564fda51e440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40116f0 tx=0x7fe7f4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fa29db3dbe02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe7f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/3137440462 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f401e060 con 0x564fda516f40 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/3137440462 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fe7f40035f0 con 0x564fda516f40 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe8137a6740 1 -- 172.21.15.47:0/3137440462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7fc00fcf0 con 0x564fda516f40 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/3137440462 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe7f40037f0 con 0x564fda516f40 2024-04-25T21:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe8137a6740 10 client.15819 fetch_fsmap learned FSMap version 33 2024-04-25T21:05:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe8137a6740 1 -- 172.21.15.47:0/3137440462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564fda530400 con 0x564fda516f40 2024-04-25T21:05:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/3137440462 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7fc00fcf0 con 0x564fda516f40 2024-04-25T21:05:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.151+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/3137440462 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7fe7f400fa40 con 0x564fda516f40 2024-04-25T21:05:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe8137a6740 10 client.15819 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T21:05:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe8137a6740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:05:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe8137a6740 20 client.15819 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe8137a6740 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564fda531910 0x564fda533d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe8137a6740 4 client.15819 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:05:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe8137a6740 1 -- 172.21.15.47:0/3137440462 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fda489390 con 0x564fda531910 2024-04-25T21:05:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564fda531910 0x564fda533d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe7f1ffb640 10 client.15819 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.152+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564fda531910 0x564fda533d00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2181dfada5011652 server_cookie=25ea30ddccb422b7 in_seq=0 out_seq=0 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/3137440462 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fda489390 con 0x564fda531910 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe7f1ffb640 10 client.15819 handle_command_reply: tid=0 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 client.15819 shutdown 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 -- 172.21.15.47:0/3137440462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516f40 msgr2=0x564fda51e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516f40 0x564fda51e440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40116f0 tx=0x7fe7f4011af0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 -- 172.21.15.47:0/3137440462 shutdown_connections 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564fda531910 0x564fda533d00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda51e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516f40 0x564fda51e440 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 --2- 172.21.15.47:0/3137440462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda516620 0x564fda521150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 -- 172.21.15.47:0/3137440462 >> 172.21.15.47:0/3137440462 conn(0x564fda512b30 msgr2=0x564fda518860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe7f1ffb640 0 client.15819 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 -- 172.21.15.47:0/3137440462 shutdown_connections 2024-04-25T21:05:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.153+0000 7fe8137a6740 1 -- 172.21.15.47:0/3137440462 wait complete. 2024-04-25T21:05:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.154+0000 7fe8137a6740 20 client.15819 trim_cache size 0 max 16384 2024-04-25T21:05:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe8137a6740 1 Processor -- start 2024-04-25T21:05:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe8137a6740 1 -- start start 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516620 0x564fda512430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda519970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 0x564fda520c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe8137a6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1adfb0 con 0x564fda516620 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe8137a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1ab880 con 0x564fda51ea70 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe8137a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1a96f0 con 0x564fda521800 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516620 0x564fda512430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe809d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda519970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516620 0x564fda512430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44704/0 (socket says 172.21.15.47:44704) 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe80ad9d640 1 -- 172.21.15.47:0/1864030087 learned_addr learned my addr 172.21.15.47:0/1864030087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe80ad9d640 1 -- 172.21.15.47:0/1864030087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 msgr2=0x564fda519970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/1864030087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda519970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe80ad9d640 1 -- 172.21.15.47:0/1864030087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 msgr2=0x564fda520c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/1864030087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 0x564fda520c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe80ad9d640 1 -- 172.21.15.47:0/1864030087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fda50e2e0 con 0x564fda516620 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.174+0000 7fe809d9b640 1 --2- 172.21.15.47:0/1864030087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda519970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.175+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/1864030087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516620 0x564fda512430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc016fd0 tx=0x7fe7fc016a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615e8d9b07ccebed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.175+0000 7fe7f3fff640 1 -- 172.21.15.47:0/1864030087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc023040 con 0x564fda516620 2024-04-25T21:05:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.175+0000 7fe7f3fff640 1 -- 172.21.15.47:0/1864030087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7fc032070 con 0x564fda516620 2024-04-25T21:05:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.175+0000 7fe7f3fff640 1 -- 172.21.15.47:0/1864030087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc01e8a0 con 0x564fda516620 2024-04-25T21:05:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 -- 172.21.15.47:0/1864030087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516620 msgr2=0x564fda512430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 --2- 172.21.15.47:0/1864030087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516620 0x564fda512430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc016fd0 tx=0x7fe7fc016a00 comp rx=0 tx=0).stop 2024-04-25T21:05:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 -- 172.21.15.47:0/1864030087 shutdown_connections 2024-04-25T21:05:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 --2- 172.21.15.47:0/1864030087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 0x564fda520c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 --2- 172.21.15.47:0/1864030087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda521800 0x564fda519970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 --2- 172.21.15.47:0/1864030087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda516620 0x564fda512430 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 -- 172.21.15.47:0/1864030087 >> 172.21.15.47:0/1864030087 conn(0x564fda512b30 msgr2=0x564fda519260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 -- 172.21.15.47:0/1864030087 shutdown_connections 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 -- 172.21.15.47:0/1864030087 wait complete. 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 -- start start 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.176+0000 7fe8137a6740 1 Processor -- start 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda524a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 0x564fda524f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe8137a6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda595e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe8137a6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1a9d40 con 0x564fda521800 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe8137a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1ab1b0 con 0x564fda51ea70 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe8137a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1aa410 con 0x564fda516620 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe8137a6740 10 client.0 _ll_register_callbacks cb 0x7fe7f4009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda595e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe809d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 0x564fda524f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe809d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 0x564fda524f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:57628/0 (socket says 172.21.15.47:57628) 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda595e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44714/0 (socket says 172.21.15.47:44714) 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe809d9b640 1 -- 172.21.15.47:0/385760150 learned_addr learned my addr 172.21.15.47:0/385760150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda524a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 -- 172.21.15.47:0/385760150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 msgr2=0x564fda524a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda524a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 -- 172.21.15.47:0/385760150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 msgr2=0x564fda524f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 0x564fda524f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 -- 172.21.15.47:0/385760150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fda596440 con 0x564fda521800 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe80ad9d640 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda595e30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc011fe0 tx=0x7fe7fc0164d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f63ff51de8f779d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.177+0000 7fe7f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/385760150 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc005020 con 0x564fda521800 2024-04-25T21:05:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/385760150 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7fe7fc0051c0 con 0x564fda521800 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 1 -- 172.21.15.47:0/385760150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7fc032070 con 0x564fda521800 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/385760150 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe7fc0053c0 con 0x564fda521800 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 10 client.15825 fetch_fsmap learned FSMap version 33 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 1 -- 172.21.15.47:0/385760150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564fda597750 con 0x564fda521800 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/385760150 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7fc032070 con 0x564fda521800 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/385760150 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7fe7fc029040 con 0x564fda521800 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 10 client.15825 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 20 client.15825 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564fda597b50 0x564fda599f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 4 client.15825 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:05:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe8137a6740 1 -- 172.21.15.47:0/385760150 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fda483200 con 0x564fda597b50 2024-04-25T21:05:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.178+0000 7fe809d9b640 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564fda597b50 0x564fda599f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.179+0000 7fe7f1ffb640 10 client.15825 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.179+0000 7fe809d9b640 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564fda597b50 0x564fda599f70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d521c749e543a7 server_cookie=380c774514a385bc in_seq=0 out_seq=0 2024-04-25T21:05:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe7f1ffb640 1 -- 172.21.15.47:0/385760150 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fda483200 con 0x564fda597b50 2024-04-25T21:05:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe7f1ffb640 10 client.15825 handle_command_reply: tid=0 2024-04-25T21:05:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 client.15825 shutdown 2024-04-25T21:05:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 -- 172.21.15.47:0/385760150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 msgr2=0x564fda595e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda595e30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc011fe0 tx=0x7fe7fc0164d0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 -- 172.21.15.47:0/385760150 shutdown_connections 2024-04-25T21:05:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x564fda597b50 0x564fda599f70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564fda521800 0x564fda595e30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe7f1ffb640 0 client.15825 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fda51ea70 0x564fda524f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 --2- 172.21.15.47:0/385760150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fda516620 0x564fda524a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 -- 172.21.15.47:0/385760150 >> 172.21.15.47:0/385760150 conn(0x564fda512b30 msgr2=0x564fda52c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 -- 172.21.15.47:0/385760150 shutdown_connections 2024-04-25T21:05:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 1 -- 172.21.15.47:0/385760150 wait complete. 2024-04-25T21:05:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.187+0000 7fe8137a6740 20 client.15825 trim_cache size 0 max 16384 2024-04-25T21:05:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:05:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:05:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:05:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:05:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 -- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fe7f80a3e40 msgr2=0x7fe7f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fe7f80a3e40 0x7fe7f80a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x564fda517bc0 tx=0x7fe80402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 -- 172.21.15.47:0/719895600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80c1010f0 msgr2=0x7fe80c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80c1010f0 0x7fe80c199300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe804009bf0 tx=0x7fe804009360 comp rx=0 tx=0).stop 2024-04-25T21:05:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 -- 172.21.15.47:0/719895600 shutdown_connections 2024-04-25T21:05:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c199dd0 0x7fe80c1a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe80c106170 0x7fe80c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fe7f80a3e40 0x7fe7f80a6300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 --2- 172.21.15.47:0/719895600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80c1010f0 0x7fe80c199300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 -- 172.21.15.47:0/719895600 >> 172.21.15.47:0/719895600 conn(0x7fe80c077fd0 msgr2=0x7fe80c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 -- 172.21.15.47:0/719895600 shutdown_connections 2024-04-25T21:05:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:00.188+0000 7fe8109ad640 1 -- 172.21.15.47:0/719895600 wait complete. 2024-04-25T21:05:01.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:00 smithi092 ceph-mon[32524]: pgmap v153: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 920 op/s 2024-04-25T21:05:01.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:00 smithi047 ceph-mon[29195]: pgmap v153: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 920 op/s 2024-04-25T21:05:01.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:00 smithi186 ceph-mon[32529]: pgmap v153: 97 pgs: 97 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 920 op/s 2024-04-25T21:05:02.381 INFO:tasks.workunit.client.0.smithi047.stdout:copying to largest ever: 0x63ef9ee 2024-04-25T21:05:03.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:02 smithi092 ceph-mon[32524]: pgmap v154: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:05:03.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:02 smithi047 ceph-mon[29195]: pgmap v154: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:05:03.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:02 smithi186 ceph-mon[32529]: pgmap v154: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:05:04.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:03 smithi047 ceph-mon[29195]: pgmap v155: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.6 MiB/s wr, 918 op/s 2024-04-25T21:05:04.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:03 smithi186 ceph-mon[32529]: pgmap v155: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.6 MiB/s wr, 918 op/s 2024-04-25T21:05:04.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:03 smithi092 ceph-mon[32524]: pgmap v155: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.6 MiB/s wr, 918 op/s 2024-04-25T21:05:06.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:06 smithi186 ceph-mon[32529]: pgmap v156: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.9 MiB/s wr, 1.39k op/s 2024-04-25T21:05:06.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:06 smithi092 ceph-mon[32524]: pgmap v156: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.9 MiB/s wr, 1.39k op/s 2024-04-25T21:05:06.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:06 smithi047 ceph-mon[29195]: pgmap v156: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.9 MiB/s wr, 1.39k op/s 2024-04-25T21:05:08.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:08 smithi047 ceph-mon[29195]: pgmap v157: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-25T21:05:08.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:08 smithi186 ceph-mon[32529]: pgmap v157: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-25T21:05:08.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:08 smithi092 ceph-mon[32524]: pgmap v157: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-25T21:05:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:10 smithi186 ceph-mon[32529]: pgmap v158: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-25T21:05:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:10 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:10 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:10 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:10.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:10 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:05:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:10 smithi092 ceph-mon[32524]: pgmap v158: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-25T21:05:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:10 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:10 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:10 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:10 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:05:10.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:10 smithi047 ceph-mon[29195]: pgmap v158: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-25T21:05:10.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:10 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:10.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:10 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:10.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:10 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:10.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:10 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:05:12.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:12 smithi186 ceph-mon[32529]: pgmap v159: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:05:12.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:12 smithi092 ceph-mon[32524]: pgmap v159: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:05:12.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:12 smithi047 ceph-mon[29195]: pgmap v159: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 -- 172.21.15.47:0/2948785593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac101ac0 msgr2=0x7f41ac105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2948785593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac101ac0 0x7f41ac105ab0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4194009900 tx=0x7f419402a410 comp rx=0 tx=0).stop 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 -- 172.21.15.47:0/2948785593 shutdown_connections 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2948785593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ac106190 0x7f41ac10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2948785593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac101ac0 0x7f41ac105ab0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2948785593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41ac101110 0x7f41ac1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 -- 172.21.15.47:0/2948785593 >> 172.21.15.47:0/2948785593 conn(0x7f41ac077fa0 msgr2=0x7f41ac0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 -- 172.21.15.47:0/2948785593 shutdown_connections 2024-04-25T21:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.299+0000 7f41b14e1640 1 -- 172.21.15.47:0/2948785593 wait complete. 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.300+0000 7f41b14e1640 1 Processor -- start 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.300+0000 7f41b14e1640 1 -- start start 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.300+0000 7f41b14e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41ac101110 0x7f41ac1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.300+0000 7f41b14e1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac106190 0x7f41ac198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.300+0000 7f41b14e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ac19cf70 0x7f41ac19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.300+0000 7f41b14e1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1110b0 con 0x7f41ac106190 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.300+0000 7f41b14e1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac110f30 con 0x7f41ac101110 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.300+0000 7f41b14e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac111230 con 0x7f41ac19cf70 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41ac101110 0x7f41ac1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac106190 0x7f41ac198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41ab7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ac19cf70 0x7f41ac19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41ac101110 0x7f41ac1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:58296/0 (socket says 172.21.15.47:58296) 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac106190 0x7f41ac198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45006/0 (socket says 172.21.15.47:45006) 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 -- 172.21.15.47:0/2669269766 learned_addr learned my addr 172.21.15.47:0/2669269766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 -- 172.21.15.47:0/2669269766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ac19cf70 msgr2=0x7f41ac19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ac19cf70 0x7f41ac19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 -- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac106190 msgr2=0x7f41ac198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac106190 0x7f41ac198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 -- 172.21.15.47:0/2669269766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac19da60 con 0x7f41ac101110 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41aaffd640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41ac101110 0x7f41ac1986b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f419c00e9f0 tx=0x7f419c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41a3fff640 1 -- 172.21.15.47:0/2669269766 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c00ccc0 con 0x7f41ac101110 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41a3fff640 1 -- 172.21.15.47:0/2669269766 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c0108e0 con 0x7f41ac101110 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41b14e1640 1 -- 172.21.15.47:0/2669269766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac19dd50 con 0x7f41ac101110 2024-04-25T21:05:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41a3fff640 1 -- 172.21.15.47:0/2669269766 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c018880 con 0x7f41ac101110 2024-04-25T21:05:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.301+0000 7f41b14e1640 1 -- 172.21.15.47:0/2669269766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac102d70 con 0x7f41ac101110 2024-04-25T21:05:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.304+0000 7f41a3fff640 1 -- 172.21.15.47:0/2669269766 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f419c010430 con 0x7f41ac101110 2024-04-25T21:05:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.304+0000 7f41b41f9740 1 Processor -- start 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.304+0000 7f41a3fff640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f41980a3dc0 0x7f41980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.304+0000 7f41b41f9740 1 -- start start 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41ab7fe640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f41980a3dc0 0x7f41980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41a3fff640 1 -- 172.21.15.47:0/2669269766 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f419c0d0ba0 con 0x7f41ac101110 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279051a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x55727905ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41b41f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557278fb6970 con 0x557279051620 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41b41f9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557278f534a0 con 0x557279051f40 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41b41f9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557278fb2930 con 0x55727905c800 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279051a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279051a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45008/0 (socket says 172.21.15.47:45008) 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aaffd640 1 -- 172.21.15.47:0/1520147414 learned_addr learned my addr 172.21.15.47:0/1520147414 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41ab7fe640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f41980a3dc0 0x7f41980a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f41a40099a0 tx=0x7f41a4012040 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41ab7fe640 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x55727905ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aa7fc640 1 -- 172.21.15.47:0/1520147414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 msgr2=0x55727905ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x55727905ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aa7fc640 1 -- 172.21.15.47:0/1520147414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 msgr2=0x557279051a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279051a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.305+0000 7f41aa7fc640 1 -- 172.21.15.47:0/1520147414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557278cc9210 con 0x557279051f40 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.306+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905c2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4194006010 tx=0x7f4194005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41cf47b417d58ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.306+0000 7f41a17fa640 1 -- 172.21.15.47:0/1520147414 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194040070 con 0x557279051f40 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.306+0000 7f41a17fa640 1 -- 172.21.15.47:0/1520147414 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4194004030 con 0x557279051f40 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.306+0000 7f41a17fa640 1 -- 172.21.15.47:0/1520147414 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194035040 con 0x557279051f40 2024-04-25T21:05:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 -- 172.21.15.47:0/1520147414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 msgr2=0x55727905c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905c2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4194006010 tx=0x7f4194005e40 comp rx=0 tx=0).stop 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 -- 172.21.15.47:0/1520147414 shutdown_connections 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x55727905ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905c2c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1520147414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279051a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 -- 172.21.15.47:0/1520147414 >> 172.21.15.47:0/1520147414 conn(0x55727904db30 msgr2=0x55727904df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 -- 172.21.15.47:0/1520147414 shutdown_connections 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 -- 172.21.15.47:0/1520147414 wait complete. 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.307+0000 7f41b41f9740 1 -- start start 2024-04-25T21:05:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41b41f9740 1 Processor -- start 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x55727905c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x5572790594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x557279059ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41b41f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55727905f3d0 con 0x557279051f40 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41b41f9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557278f510b0 con 0x55727905c800 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41b41f9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55727905f550 con 0x557279051620 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41b41f9740 10 client.0 _ll_register_callbacks cb 0x557278f3ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x55727905c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x5572790594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41ab7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x557279059ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x55727905c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:35832/0 (socket says 172.21.15.47:35832) 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x5572790594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45022/0 (socket says 172.21.15.47:45022) 2024-04-25T21:05:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41ab7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x557279059ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:58314/0 (socket says 172.21.15.47:58314) 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aaffd640 1 -- 172.21.15.47:0/4259240359 learned_addr learned my addr 172.21.15.47:0/4259240359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aaffd640 1 -- 172.21.15.47:0/4259240359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 msgr2=0x557279059ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aaffd640 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x557279059ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aaffd640 1 -- 172.21.15.47:0/4259240359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 msgr2=0x5572790594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aaffd640 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x5572790594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.308+0000 7f41aaffd640 1 -- 172.21.15.47:0/4259240359 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557278d95620 con 0x557279051620 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41aaffd640 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x55727905c1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f419c0ce900 tx=0x7f419c09d250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b031efc67c49509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41927fc640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41927fc640 1 -- 172.21.15.47:0/4259240359 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0a57c0 con 0x557279051620 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41927fc640 1 -- 172.21.15.47:0/4259240359 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f419c010a90 con 0x557279051620 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41b41f9740 1 -- 172.21.15.47:0/4259240359 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4194004030 con 0x557279051620 2024-04-25T21:05:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41927fc640 1 -- 172.21.15.47:0/4259240359 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f419c0991c0 con 0x557279051620 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41b41f9740 10 client.25453 fetch_fsmap learned FSMap version 33 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41b41f9740 1 -- 172.21.15.47:0/4259240359 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55727906b380 con 0x557279051620 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.309+0000 7f41927fc640 1 -- 172.21.15.47:0/4259240359 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4194004030 con 0x557279051620 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41927fc640 1 -- 172.21.15.47:0/4259240359 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7f419c09ba10 con 0x557279051620 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41b41f9740 10 client.25453 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41b41f9740 10 client.25453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41b41f9740 20 client.25453 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41b41f9740 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55727906c7f0 0x55727906ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41b41f9740 4 client.25453 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41b41f9740 1 -- 172.21.15.47:0/4259240359 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557278fc4390 con 0x55727906c7f0 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55727906c7f0 0x55727906ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41927fc640 10 client.25453 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.310+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55727906c7f0 0x55727906ebe0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c9fe1ad16eb4b1 server_cookie=d017e946634be7d2 in_seq=0 out_seq=0 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41927fc640 1 -- 172.21.15.47:0/4259240359 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557278fc4390 con 0x55727906c7f0 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41927fc640 10 client.25453 handle_command_reply: tid=0 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 client.25453 shutdown 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 -- 172.21.15.47:0/4259240359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 msgr2=0x55727905c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x55727905c1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f419c0ce900 tx=0x7f419c09d250 comp rx=0 tx=0).stop 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 -- 172.21.15.47:0/4259240359 shutdown_connections 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x55727906c7f0 0x55727906ebe0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x557279059ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x5572790594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41927fc640 0 client.25453 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 --2- 172.21.15.47:0/4259240359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x55727905c1e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 -- 172.21.15.47:0/4259240359 >> 172.21.15.47:0/4259240359 conn(0x55727904db30 msgr2=0x5572790538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 -- 172.21.15.47:0/4259240359 shutdown_connections 2024-04-25T21:05:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 1 -- 172.21.15.47:0/4259240359 wait complete. 2024-04-25T21:05:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.311+0000 7f41b41f9740 20 client.25453 trim_cache size 0 max 16384 2024-04-25T21:05:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41b41f9740 1 Processor -- start 2024-04-25T21:05:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41b41f9740 1 -- start start 2024-04-25T21:05:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279068220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x557279058db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41b41f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1ae1f0 con 0x557279051620 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41b41f9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1abac0 con 0x557279051f40 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41b41f9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1a9930 con 0x55727905c800 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279068220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41ab7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x557279058db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279068220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45026/0 (socket says 172.21.15.47:45026) 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41ab7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x557279058db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:35846/0 (socket says 172.21.15.47:35846) 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41aaffd640 1 -- 172.21.15.47:0/3604763552 learned_addr learned my addr 172.21.15.47:0/3604763552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.332+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41aaffd640 1 -- 172.21.15.47:0/3604763552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 msgr2=0x557279058db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41aaffd640 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x557279058db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41aaffd640 1 -- 172.21.15.47:0/3604763552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 msgr2=0x55727905bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41aaffd640 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41aaffd640 1 -- 172.21.15.47:0/3604763552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727906ef50 con 0x557279051620 2024-04-25T21:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41aaffd640 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279068220 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f419c09cd10 tx=0x7f419c09b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6b3a7dbff083f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41a0ff9640 1 -- 172.21.15.47:0/3604763552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0a57c0 con 0x557279051620 2024-04-25T21:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41a0ff9640 1 -- 172.21.15.47:0/3604763552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c0bd0b0 con 0x557279051620 2024-04-25T21:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.333+0000 7f41a0ff9640 1 -- 172.21.15.47:0/3604763552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0988a0 con 0x557279051620 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 -- 172.21.15.47:0/3604763552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 msgr2=0x557279068220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279068220 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f419c09cd10 tx=0x7f419c09b060 comp rx=0 tx=0).stop 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 -- 172.21.15.47:0/3604763552 shutdown_connections 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55727905c800 0x557279058db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557279051f40 0x55727905bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 --2- 172.21.15.47:0/3604763552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051620 0x557279068220 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 -- 172.21.15.47:0/3604763552 >> 172.21.15.47:0/3604763552 conn(0x55727904db30 msgr2=0x557279067b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 -- 172.21.15.47:0/3604763552 shutdown_connections 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 -- 172.21.15.47:0/3604763552 wait complete. 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 -- start start 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.334+0000 7f41b41f9740 1 Processor -- start 2024-04-25T21:05:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x557279064bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x557279061f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41b41f9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x5572790624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41b41f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1a9f80 con 0x557279051f40 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41b41f9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1ab3f0 con 0x55727905c800 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41b41f9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1aa650 con 0x557279051620 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41b41f9740 10 client.0 _ll_register_callbacks cb 0x7f419c09c4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41aa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x557279061f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x557279064bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41ab7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x5572790624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41aaffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x557279064bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.47:35860/0 (socket says 172.21.15.47:35860) 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41aa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x557279061f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45038/0 (socket says 172.21.15.47:45038) 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41aaffd640 1 -- 172.21.15.47:0/1887759354 learned_addr learned my addr 172.21.15.47:0/1887759354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41ab7fe640 1 -- 172.21.15.47:0/1887759354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 msgr2=0x557279064bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41ab7fe640 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x557279064bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41ab7fe640 1 -- 172.21.15.47:0/1887759354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 msgr2=0x557279061f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41ab7fe640 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x557279061f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41ab7fe640 1 -- 172.21.15.47:0/1887759354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557279062cf0 con 0x55727905c800 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41aaffd640 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x557279064bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41aa7fc640 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x557279061f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41ab7fe640 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x5572790624c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009b80 tx=0x7f41a4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a500a274055307f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.335+0000 7f41927fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41927fc640 1 -- 172.21.15.47:0/1887759354 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4003cf0 con 0x55727905c800 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41927fc640 1 -- 172.21.15.47:0/1887759354 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f41a4034470 con 0x55727905c800 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 1 -- 172.21.15.47:0/1887759354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f419c0bd0b0 con 0x55727905c800 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41927fc640 1 -- 172.21.15.47:0/1887759354 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f41a4003e90 con 0x55727905c800 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 10 client.25463 fetch_fsmap learned FSMap version 33 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 1 -- 172.21.15.47:0/1887759354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5572790d2e70 con 0x55727905c800 2024-04-25T21:05:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41927fc640 1 -- 172.21.15.47:0/1887759354 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c0bd0b0 con 0x55727905c800 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41927fc640 1 -- 172.21.15.47:0/1887759354 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7f41a402b540 con 0x55727905c800 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 10 client.25463 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 10 client.25463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 20 client.25463 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5572790d35d0 0x5572790d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 4 client.25463 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.336+0000 7f41b41f9740 1 -- 172.21.15.47:0/1887759354 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557278fbe200 con 0x5572790d35d0 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.337+0000 7f41aaffd640 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5572790d35d0 0x5572790d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.337+0000 7f41927fc640 10 client.25463 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.337+0000 7f41aaffd640 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5572790d35d0 0x5572790d59f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da24be21dbf3706 server_cookie=e7388522c27ae2c8 in_seq=0 out_seq=0 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.345+0000 7f41927fc640 1 -- 172.21.15.47:0/1887759354 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557278fbe200 con 0x5572790d35d0 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.345+0000 7f41927fc640 10 client.25463 handle_command_reply: tid=0 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.345+0000 7f41b41f9740 1 client.25463 shutdown 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.345+0000 7f41b41f9740 1 -- 172.21.15.47:0/1887759354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 msgr2=0x5572790624c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.345+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x5572790624c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009b80 tx=0x7f41a4004a40 comp rx=0 tx=0).stop 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.345+0000 7f41b41f9740 1 -- 172.21.15.47:0/1887759354 shutdown_connections 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.345+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5572790d35d0 0x5572790d59f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.346+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55727905c800 0x5572790624c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.346+0000 7f41927fc640 0 client.25463 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.346+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557279051f40 0x557279061f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.346+0000 7f41b41f9740 1 --2- 172.21.15.47:0/1887759354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557279051620 0x557279064bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.346+0000 7f41b41f9740 1 -- 172.21.15.47:0/1887759354 >> 172.21.15.47:0/1887759354 conn(0x55727904db30 msgr2=0x55727905ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.346+0000 7f41b41f9740 1 -- 172.21.15.47:0/1887759354 shutdown_connections 2024-04-25T21:05:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.347+0000 7f41b41f9740 1 -- 172.21.15.47:0/1887759354 wait complete. 2024-04-25T21:05:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.347+0000 7f41b41f9740 20 client.25463 trim_cache size 0 max 16384 2024-04-25T21:05:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:05:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:05:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:05:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 -- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f41980a3dc0 msgr2=0x7f41980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f41980a3dc0 0x7f41980a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f41a40099a0 tx=0x7f41a4012040 comp rx=0 tx=0).stop 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 -- 172.21.15.47:0/2669269766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41ac101110 msgr2=0x7f41ac1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41ac101110 0x7f41ac1986b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f419c00e9f0 tx=0x7f419c00edf0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 -- 172.21.15.47:0/2669269766 shutdown_connections 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ac19cf70 0x7f41ac19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41ac106190 0x7f41ac198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f41980a3dc0 0x7f41980a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 --2- 172.21.15.47:0/2669269766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41ac101110 0x7f41ac1986b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 -- 172.21.15.47:0/2669269766 >> 172.21.15.47:0/2669269766 conn(0x7f41ac077fa0 msgr2=0x7f41ac06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 -- 172.21.15.47:0/2669269766 shutdown_connections 2024-04-25T21:05:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:13.348+0000 7f41b14e1640 1 -- 172.21.15.47:0/2669269766 wait complete. 2024-04-25T21:05:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:14 smithi186 ceph-mon[32529]: pgmap v160: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.6 MiB/s wr, 915 op/s 2024-04-25T21:05:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:14 smithi092 ceph-mon[32524]: pgmap v160: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.6 MiB/s wr, 915 op/s 2024-04-25T21:05:14.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:14 smithi047 ceph-mon[29195]: pgmap v160: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.6 MiB/s wr, 915 op/s 2024-04-25T21:05:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:16 smithi186 ceph-mon[32529]: pgmap v161: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.8 MiB/s wr, 1.35k op/s 2024-04-25T21:05:16.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:16 smithi092 ceph-mon[32524]: pgmap v161: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.8 MiB/s wr, 1.35k op/s 2024-04-25T21:05:16.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:16 smithi047 ceph-mon[29195]: pgmap v161: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.8 MiB/s wr, 1.35k op/s 2024-04-25T21:05:18.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:18 smithi186 ceph-mon[32529]: pgmap v162: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T21:05:18.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:18 smithi092 ceph-mon[32524]: pgmap v162: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T21:05:18.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:18 smithi047 ceph-mon[29195]: pgmap v162: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T21:05:20.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:20 smithi186 ceph-mon[32529]: pgmap v163: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T21:05:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:20 smithi092 ceph-mon[32524]: pgmap v163: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T21:05:20.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:20 smithi047 ceph-mon[29195]: pgmap v163: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T21:05:22.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:22 smithi186 ceph-mon[32529]: pgmap v164: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.7 MiB/s wr, 1.30k op/s 2024-04-25T21:05:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:22 smithi092 ceph-mon[32524]: pgmap v164: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.7 MiB/s wr, 1.30k op/s 2024-04-25T21:05:22.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:22 smithi047 ceph-mon[29195]: pgmap v164: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.7 MiB/s wr, 1.30k op/s 2024-04-25T21:05:23.165 INFO:tasks.workunit.client.0.smithi047.stdout:All 50000 operations completed A-OK! 2024-04-25T21:05:23.166 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T21:05:23.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T21:05:24.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:24 smithi186 ceph-mon[32529]: pgmap v165: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.5 MiB/s wr, 861 op/s 2024-04-25T21:05:24.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:24 smithi092 ceph-mon[32524]: pgmap v165: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.5 MiB/s wr, 861 op/s 2024-04-25T21:05:24.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:24 smithi047 ceph-mon[29195]: pgmap v165: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.5 MiB/s wr, 861 op/s 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 -- 172.21.15.47:0/1043217599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8102b70 msgr2=0x7ff9a8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/1043217599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8102b70 0x7ff9a8106bb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff990009900 tx=0x7ff99002a410 comp rx=0 tx=0).stop 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 -- 172.21.15.47:0/1043217599 shutdown_connections 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/1043217599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a8107290 0x7ff9a810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/1043217599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8102b70 0x7ff9a8106bb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/1043217599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a81021c0 0x7ff9a81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 -- 172.21.15.47:0/1043217599 >> 172.21.15.47:0/1043217599 conn(0x7ff9a80fda20 msgr2=0x7ff9a80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 -- 172.21.15.47:0/1043217599 shutdown_connections 2024-04-25T21:05:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.441+0000 7ff9ace4e640 1 -- 172.21.15.47:0/1043217599 wait complete. 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9ace4e640 1 Processor -- start 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9ace4e640 1 -- start start 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9ace4e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a81021c0 0x7ff9a8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9ace4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a8107290 0x7ff9a8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9ace4e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8074cd0 0x7ff9a81a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9ace4e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8112230 con 0x7ff9a8074cd0 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9ace4e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81120b0 con 0x7ff9a8107290 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9ace4e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81123b0 con 0x7ff9a81021c0 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9a6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8074cd0 0x7ff9a81a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a8107290 0x7ff9a8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a8107290 0x7ff9a8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:33694/0 (socket says 172.21.15.47:33694) 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff99ffff640 1 -- 172.21.15.47:0/624670769 learned_addr learned my addr 172.21.15.47:0/624670769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.442+0000 7ff9a6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8074cd0 0x7ff9a81a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49658/0 (socket says 172.21.15.47:49658) 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff9a659c640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a81021c0 0x7ff9a8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99ffff640 1 -- 172.21.15.47:0/624670769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a81021c0 msgr2=0x7ff9a8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99ffff640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a81021c0 0x7ff9a8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99ffff640 1 -- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8074cd0 msgr2=0x7ff9a81a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99ffff640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8074cd0 0x7ff9a81a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99ffff640 1 -- 172.21.15.47:0/624670769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a81a6fe0 con 0x7ff9a8107290 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8074cd0 0x7ff9a81a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99ffff640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a8107290 0x7ff9a8074790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff990006010 tx=0x7ff990004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99f7fe640 1 -- 172.21.15.47:0/624670769 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99003f030 con 0x7ff9a8107290 2024-04-25T21:05:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff9ace4e640 1 -- 172.21.15.47:0/624670769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a81a7270 con 0x7ff9a8107290 2024-04-25T21:05:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99f7fe640 1 -- 172.21.15.47:0/624670769 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff990003aa0 con 0x7ff9a8107290 2024-04-25T21:05:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff99f7fe640 1 -- 172.21.15.47:0/624670769 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990035070 con 0x7ff9a8107290 2024-04-25T21:05:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.443+0000 7ff9ace4e640 1 -- 172.21.15.47:0/624670769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a81a77b0 con 0x7ff9a8107290 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9afb64740 1 Processor -- start 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff99f7fe640 1 -- 172.21.15.47:0/624670769 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff990042030 con 0x7ff9a8107290 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9afb64740 1 -- start start 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 0x5594b9549a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b95542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b9556bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9afb64740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5594b94ae970 con 0x5594b9554800 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9afb64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5594b944b4a0 con 0x5594b9549f40 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9afb64740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5594b94aa930 con 0x5594b9549620 2024-04-25T21:05:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9a659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b95542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b9556bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9a659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b95542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.47:33710/0 (socket says 172.21.15.47:33710) 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9a659c640 1 -- 172.21.15.47:0/1557089877 learned_addr learned my addr 172.21.15.47:0/1557089877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/1557089877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 0x5594b9549a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff99f7fe640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9940a3df0 0x7ff9940a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9a659c640 1 -- 172.21.15.47:0/1557089877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 msgr2=0x5594b9549a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9a659c640 1 --2- 172.21.15.47:0/1557089877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 0x5594b9549a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9a659c640 1 -- 172.21.15.47:0/1557089877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 msgr2=0x5594b9556bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.446+0000 7ff9a659c640 1 --2- 172.21.15.47:0/1557089877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b9556bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.447+0000 7ff9a659c640 1 -- 172.21.15.47:0/1557089877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b91c1210 con 0x5594b9549f40 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.447+0000 7ff99f7fe640 1 -- 172.21.15.47:0/624670769 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ff9900f0b40 con 0x7ff9a8107290 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.447+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9940a3df0 0x7ff9940a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.447+0000 7ff9a659c640 1 --2- 172.21.15.47:0/1557089877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b95542c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff998009cf0 tx=0x7ff998011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd26d6722d4622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.447+0000 7ff99cff9640 1 -- 172.21.15.47:0/1557089877 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998004230 con 0x5594b9549f40 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.447+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9940a3df0 0x7ff9940a62b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000a880 tx=0x7ff9a0004b50 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.447+0000 7ff99cff9640 1 -- 172.21.15.47:0/1557089877 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9980043d0 con 0x5594b9549f40 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.447+0000 7ff99cff9640 1 -- 172.21.15.47:0/1557089877 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998019880 con 0x5594b9549f40 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- 172.21.15.47:0/1557089877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 msgr2=0x5594b95542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 --2- 172.21.15.47:0/1557089877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b95542c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff998009cf0 tx=0x7ff998011810 comp rx=0 tx=0).stop 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- 172.21.15.47:0/1557089877 shutdown_connections 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 --2- 172.21.15.47:0/1557089877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b9556bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 --2- 172.21.15.47:0/1557089877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b95542c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 --2- 172.21.15.47:0/1557089877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 0x5594b9549a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- 172.21.15.47:0/1557089877 >> 172.21.15.47:0/1557089877 conn(0x5594b9545b30 msgr2=0x5594b9545f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- 172.21.15.47:0/1557089877 shutdown_connections 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- 172.21.15.47:0/1557089877 wait complete. 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- start start 2024-04-25T21:05:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 Processor -- start 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 0x5594b9554150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b9551440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5594b9557430 con 0x5594b9554800 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5594b94490b0 con 0x5594b9549f40 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5594b95575b0 con 0x5594b9549620 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.448+0000 7ff9afb64740 10 client.0 _ll_register_callbacks cb 0x5594b9436d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b9551440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49682/0 (socket says 172.21.15.47:49682) 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 -- 172.21.15.47:0/3047672391 learned_addr learned my addr 172.21.15.47:0/3047672391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a659c640 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 0x5594b9554150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 -- 172.21.15.47:0/3047672391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 msgr2=0x5594b9554150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 0x5594b9554150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 -- 172.21.15.47:0/3047672391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 msgr2=0x5594b9551440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b9551440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 -- 172.21.15.47:0/3047672391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b928d620 con 0x5594b9554800 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff99ffff640 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b9551440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95519a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00166a0 tx=0x7ff9a000d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ae23d520ad14ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff98dffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff98dffb640 1 -- 172.21.15.47:0/3047672391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0027060 con 0x5594b9554800 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff98dffb640 1 -- 172.21.15.47:0/3047672391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ff9a0015760 con 0x5594b9554800 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9afb64740 1 -- 172.21.15.47:0/3047672391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9980043d0 con 0x5594b9554800 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff98dffb640 1 -- 172.21.15.47:0/3047672391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9a0015960 con 0x5594b9554800 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9afb64740 10 client.15855 fetch_fsmap learned FSMap version 34 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.449+0000 7ff9afb64740 1 -- 172.21.15.47:0/3047672391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5594b95633c0 con 0x5594b9554800 2024-04-25T21:05:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff98dffb640 1 -- 172.21.15.47:0/3047672391 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9980043d0 con 0x5594b9554800 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff98dffb640 1 -- 172.21.15.47:0/3047672391 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7ff9a002a3a0 con 0x5594b9554800 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff9afb64740 10 client.15855 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff9afb64740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff9afb64740 20 client.15855 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5594b9564880 0x5594b9566cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff9afb64740 4 client.15855 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff9afb64740 1 -- 172.21.15.47:0/3047672391 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594b94bc390 con 0x5594b9564880 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff9a659c640 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5594b9564880 0x5594b9566cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff98dffb640 10 client.15855 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.450+0000 7ff9a659c640 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5594b9564880 0x5594b9566cb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e0abad62cf32e5 server_cookie=d6adedcb0ef90a60 in_seq=0 out_seq=0 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff98dffb640 1 -- 172.21.15.47:0/3047672391 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594b94bc390 con 0x5594b9564880 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff98dffb640 10 client.15855 handle_command_reply: tid=0 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 client.15855 shutdown 2024-04-25T21:05:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 -- 172.21.15.47:0/3047672391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 msgr2=0x5594b95519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95519a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00166a0 tx=0x7ff9a000d040 comp rx=0 tx=0).stop 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 -- 172.21.15.47:0/3047672391 shutdown_connections 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5594b9564880 0x5594b9566cb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95519a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549f40 0x5594b9551440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff98dffb640 0 client.15855 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3047672391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549620 0x5594b9554150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 -- 172.21.15.47:0/3047672391 >> 172.21.15.47:0/3047672391 conn(0x5594b9545b30 msgr2=0x5594b954b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 -- 172.21.15.47:0/3047672391 shutdown_connections 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.451+0000 7ff9afb64740 1 -- 172.21.15.47:0/3047672391 wait complete. 2024-04-25T21:05:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.452+0000 7ff9afb64740 20 client.15855 trim_cache size 0 max 16384 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.466+0000 7ff9afb64740 1 Processor -- start 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9afb64740 1 -- start start 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 0x5594b95602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9549f40 0x5594b9544b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9554800 0x5594b956c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9afb64740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81b2670 con 0x5594b9549f40 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9afb64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81aff10 con 0x5594b9549620 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9afb64740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81add80 con 0x5594b9554800 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9549f40 0x5594b9544b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9549f40 0x5594b9544b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49686/0 (socket says 172.21.15.47:49686) 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 -- 172.21.15.47:0/3034030090 learned_addr learned my addr 172.21.15.47:0/3034030090 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9a659c640 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 0x5594b95602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9554800 0x5594b956c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 -- 172.21.15.47:0/3034030090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9554800 msgr2=0x5594b956c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9554800 0x5594b956c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 -- 172.21.15.47:0/3034030090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 msgr2=0x5594b95602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 0x5594b95602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 -- 172.21.15.47:0/3034030090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b9551d10 con 0x5594b9549f40 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff99ffff640 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9549f40 0x5594b9544b30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff9900bc8a0 tx=0x7ff9900f4690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635c13dd56c9830b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.467+0000 7ff98ffff640 1 -- 172.21.15.47:0/3034030090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990049040 con 0x5594b9549f40 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff98ffff640 1 -- 172.21.15.47:0/3034030090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99003b460 con 0x5594b9549f40 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff98ffff640 1 -- 172.21.15.47:0/3034030090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990003c50 con 0x5594b9549f40 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 -- 172.21.15.47:0/3034030090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9549f40 msgr2=0x5594b9544b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9549f40 0x5594b9544b30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff9900bc8a0 tx=0x7ff9900f4690 comp rx=0 tx=0).stop 2024-04-25T21:05:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 -- 172.21.15.47:0/3034030090 shutdown_connections 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9554800 0x5594b956c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9549f40 0x5594b9544b30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3034030090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 0x5594b95602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 -- 172.21.15.47:0/3034030090 >> 172.21.15.47:0/3034030090 conn(0x5594b9545b30 msgr2=0x5594b9544420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 -- 172.21.15.47:0/3034030090 shutdown_connections 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 -- 172.21.15.47:0/3034030090 wait complete. 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 -- start start 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.468+0000 7ff9afb64740 1 Processor -- start 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 0x5594b955a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549f40 0x5594b955ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9afb64740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9afb64740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81ae3d0 con 0x5594b9554800 2024-04-25T21:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9afb64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81af840 con 0x5594b9549620 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9afb64740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81aeaa0 con 0x5594b9549f40 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9afb64740 10 client.0 _ll_register_callbacks cb 0x7ff9a000a620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 0x5594b955a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49696/0 (socket says 172.21.15.47:49696) 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 -- 172.21.15.47:0/3466447722 learned_addr learned my addr 172.21.15.47:0/3466447722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff99ffff640 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549f40 0x5594b955ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 -- 172.21.15.47:0/3466447722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549f40 msgr2=0x5594b955ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549f40 0x5594b955ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 -- 172.21.15.47:0/3466447722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 msgr2=0x5594b955a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 0x5594b955a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 -- 172.21.15.47:0/3466447722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b95ca380 con 0x5594b9554800 2024-04-25T21:05:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9a6d9d640 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95c9e40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0007d50 tx=0x7ff9a00077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455532eea3b21228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff98dffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff98dffb640 1 -- 172.21.15.47:0/3466447722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0037030 con 0x5594b9554800 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff98dffb640 1 -- 172.21.15.47:0/3466447722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ff9a0015b00 con 0x5594b9554800 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.469+0000 7ff9afb64740 1 -- 172.21.15.47:0/3466447722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5594b94b6200 con 0x5594b9554800 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff98dffb640 1 -- 172.21.15.47:0/3466447722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9a0027020 con 0x5594b9554800 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9afb64740 10 client.15861 fetch_fsmap learned FSMap version 34 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9afb64740 1 -- 172.21.15.47:0/3466447722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5594b95cb630 con 0x5594b9554800 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff98dffb640 1 -- 172.21.15.47:0/3466447722 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5594b94b6200 con 0x5594b9554800 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff98dffb640 1 -- 172.21.15.47:0/3466447722 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7ff9a0022950 con 0x5594b9554800 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9afb64740 10 client.15861 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9afb64740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9afb64740 20 client.15861 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5594b95cbe30 0x5594b95ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9afb64740 4 client.15861 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:05:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9afb64740 1 -- 172.21.15.47:0/3466447722 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594b94325f0 con 0x5594b95cbe30 2024-04-25T21:05:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9a659c640 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5594b95cbe30 0x5594b95ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff98dffb640 10 client.15861 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.470+0000 7ff9a659c640 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5594b95cbe30 0x5594b95ce250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a4a0af7dd58bf server_cookie=f71cd6352ff71b94 in_seq=0 out_seq=0 2024-04-25T21:05:26.477 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:26 smithi186 ceph-mon[32529]: pgmap v166: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.4 MiB/s wr, 1.16k op/s 2024-04-25T21:05:26.477 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:26 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.478+0000 7ff98dffb640 1 -- 172.21.15.47:0/3466447722 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594b94325f0 con 0x5594b95cbe30 2024-04-25T21:05:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff98dffb640 10 client.15861 handle_command_reply: tid=0 2024-04-25T21:05:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 client.15861 shutdown 2024-04-25T21:05:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 -- 172.21.15.47:0/3466447722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 msgr2=0x5594b95c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95c9e40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0007d50 tx=0x7ff9a00077f0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 -- 172.21.15.47:0/3466447722 shutdown_connections 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5594b95cbe30 0x5594b95ce250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594b9554800 0x5594b95c9e40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff98dffb640 0 client.15861 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594b9549f40 0x5594b955ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 --2- 172.21.15.47:0/3466447722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594b9549620 0x5594b955a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 -- 172.21.15.47:0/3466447722 >> 172.21.15.47:0/3466447722 conn(0x5594b9545b30 msgr2=0x5594b9544cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 -- 172.21.15.47:0/3466447722 shutdown_connections 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 1 -- 172.21.15.47:0/3466447722 wait complete. 2024-04-25T21:05:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9afb64740 20 client.15861 trim_cache size 0 max 16384 2024-04-25T21:05:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:05:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:05:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:05:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:05:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 -- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9940a3df0 msgr2=0x7ff9940a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9940a3df0 0x7ff9940a62b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000a880 tx=0x7ff9a0004b50 comp rx=0 tx=0).stop 2024-04-25T21:05:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 -- 172.21.15.47:0/624670769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a8107290 msgr2=0x7ff9a8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a8107290 0x7ff9a8074790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff990006010 tx=0x7ff990004060 comp rx=0 tx=0).stop 2024-04-25T21:05:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 -- 172.21.15.47:0/624670769 shutdown_connections 2024-04-25T21:05:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9a8074cd0 0x7ff9a81a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff9940a3df0 0x7ff9940a62b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a8107290 0x7ff9a8074790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 --2- 172.21.15.47:0/624670769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a81021c0 0x7ff9a8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.479+0000 7ff9ace4e640 1 -- 172.21.15.47:0/624670769 >> 172.21.15.47:0/624670769 conn(0x7ff9a80fda20 msgr2=0x7ff9a80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.480+0000 7ff9ace4e640 1 -- 172.21.15.47:0/624670769 shutdown_connections 2024-04-25T21:05:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:26.480+0000 7ff9ace4e640 1 -- 172.21.15.47:0/624670769 wait complete. 2024-04-25T21:05:26.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:26 smithi092 ceph-mon[32524]: pgmap v166: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.4 MiB/s wr, 1.16k op/s 2024-04-25T21:05:26.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:26 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:26.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:26 smithi047 ceph-mon[29195]: pgmap v166: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.4 MiB/s wr, 1.16k op/s 2024-04-25T21:05:26.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:26 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:28.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:28 smithi186 ceph-mon[32529]: pgmap v167: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 728 op/s 2024-04-25T21:05:28.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:28 smithi092 ceph-mon[32524]: pgmap v167: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 728 op/s 2024-04-25T21:05:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:28 smithi047 ceph-mon[29195]: pgmap v167: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 728 op/s 2024-04-25T21:05:30.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:30 smithi186 ceph-mon[32529]: pgmap v168: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 728 op/s 2024-04-25T21:05:30.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:30 smithi092 ceph-mon[32524]: pgmap v168: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 728 op/s 2024-04-25T21:05:30.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:30 smithi047 ceph-mon[29195]: pgmap v168: 97 pgs: 97 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 728 op/s 2024-04-25T21:05:32.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:32 smithi186 ceph-mon[32529]: pgmap v169: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.2 MiB/s wr, 857 op/s 2024-04-25T21:05:32.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:32 smithi092 ceph-mon[32524]: pgmap v169: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.2 MiB/s wr, 857 op/s 2024-04-25T21:05:32.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:32 smithi047 ceph-mon[29195]: pgmap v169: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.2 MiB/s wr, 857 op/s 2024-04-25T21:05:34.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:34 smithi186 ceph-mon[32529]: pgmap v170: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 429 op/s 2024-04-25T21:05:34.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:34 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:34.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:34 smithi092 ceph-mon[32524]: pgmap v170: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 429 op/s 2024-04-25T21:05:34.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:34 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:34.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:34 smithi047 ceph-mon[29195]: pgmap v170: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 429 op/s 2024-04-25T21:05:34.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:34 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:36.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:36 smithi186 ceph-mon[32529]: pgmap v171: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.2 MiB/s wr, 583 op/s 2024-04-25T21:05:36.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:36 smithi092 ceph-mon[32524]: pgmap v171: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.2 MiB/s wr, 583 op/s 2024-04-25T21:05:36.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:36 smithi047 ceph-mon[29195]: pgmap v171: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.2 MiB/s wr, 583 op/s 2024-04-25T21:05:37.549 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-25T21:05:37.549 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T21:05:37.855 DEBUG:teuthology.parallel:result is None 2024-04-25T21:05:37.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T21:05:37.897 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T21:05:37.897 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T21:05:37.909 INFO:tasks.quiescer:joining Quiescers 2024-04-25T21:05:37.909 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-25T21:05:37.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:05:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:38.453 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:38 smithi047 ceph-mon[29195]: pgmap v172: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-04-25T21:05:38.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:38 smithi186 ceph-mon[32529]: pgmap v172: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-04-25T21:05:38.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:38 smithi092 ceph-mon[32524]: pgmap v172: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-04-25T21:05:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 -- 172.21.15.47:0/3413459131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 msgr2=0x7f2da41035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 --2- 172.21.15.47:0/3413459131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 0x7f2da41035e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009900 tx=0x7f2da002a480 comp rx=0 tx=0).stop 2024-04-25T21:05:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 -- 172.21.15.47:0/3413459131 shutdown_connections 2024-04-25T21:05:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 --2- 172.21.15.47:0/3413459131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2da410c5d0 0x7f2da410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 --2- 172.21.15.47:0/3413459131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da4103b20 0x7f2da410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 --2- 172.21.15.47:0/3413459131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 0x7f2da41035e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 -- 172.21.15.47:0/3413459131 >> 172.21.15.47:0/3413459131 conn(0x7f2da40771d0 msgr2=0x7f2da40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 -- 172.21.15.47:0/3413459131 shutdown_connections 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.574+0000 7f2daba12640 1 -- 172.21.15.47:0/3413459131 wait complete. 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2daba12640 1 Processor -- start 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2daba12640 1 -- start start 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2daba12640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 0x7f2da4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2daba12640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da4103b20 0x7f2da4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2daba12640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2da410c5d0 0x7f2da419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2daba12640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4112220 con 0x7f2da4103200 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2daba12640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41120a0 con 0x7f2da4103b20 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2daba12640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41123a0 con 0x7f2da410c5d0 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2da97ae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 0x7f2da4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2da97ae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 0x7f2da4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47838/0 (socket says 172.21.15.47:47838) 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.575+0000 7f2da97ae640 1 -- 172.21.15.47:0/2458338992 learned_addr learned my addr 172.21.15.47:0/2458338992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2da9faf640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2da410c5d0 0x7f2da419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2da8fad640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da4103b20 0x7f2da4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2da97ae640 1 -- 172.21.15.47:0/2458338992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2da410c5d0 msgr2=0x7f2da419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2da97ae640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2da410c5d0 0x7f2da419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2da97ae640 1 -- 172.21.15.47:0/2458338992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da4103b20 msgr2=0x7f2da4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2da97ae640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da4103b20 0x7f2da4198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2da97ae640 1 -- 172.21.15.47:0/2458338992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da419d6a0 con 0x7f2da4103200 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2da97ae640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 0x7f2da4198700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2da0005940 tx=0x7f2da00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2d9a7fc640 1 -- 172.21.15.47:0/2458338992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da003f030 con 0x7f2da4103200 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2daba12640 1 -- 172.21.15.47:0/2458338992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da419d930 con 0x7f2da4103200 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2d9a7fc640 1 -- 172.21.15.47:0/2458338992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da0003cb0 con 0x7f2da4103200 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2d9a7fc640 1 -- 172.21.15.47:0/2458338992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0035070 con 0x7f2da4103200 2024-04-25T21:05:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.576+0000 7f2daba12640 1 -- 172.21.15.47:0/2458338992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41a5220 con 0x7f2da4103200 2024-04-25T21:05:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.578+0000 7f2d9a7fc640 1 -- 172.21.15.47:0/2458338992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f2da0042030 con 0x7f2da4103200 2024-04-25T21:05:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2dae7fd740 1 Processor -- start 2024-04-25T21:05:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2dae7fd740 1 -- start start 2024-04-25T21:05:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dadfa30 0x555e7dadfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 0x555e7daea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 0x555e7daed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555e7da40970 con 0x555e7dadfa30 2024-04-25T21:05:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555e7d9dd4a0 con 0x555e7dae0350 2024-04-25T21:05:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555e7da3c930 con 0x555e7daeac10 2024-04-25T21:05:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2d9a7fc640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2d900a3dc0 0x7f2d900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2da9faf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dadfa30 0x555e7dadfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2da9faf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dadfa30 0x555e7dadfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47854/0 (socket says 172.21.15.47:47854) 2024-04-25T21:05:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2da9faf640 1 -- 172.21.15.47:0/3436025843 learned_addr learned my addr 172.21.15.47:0/3436025843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2da8fad640 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 0x555e7daed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2da9faf640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2d900a3dc0 0x7f2d900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2d9a7fc640 1 -- 172.21.15.47:0/2458338992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f2da00f0da0 con 0x7f2da4103200 2024-04-25T21:05:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.579+0000 7f2da97ae640 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 0x555e7daea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2da9faf640 1 -- 172.21.15.47:0/3436025843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 msgr2=0x555e7daed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2da9faf640 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 0x555e7daed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2da9faf640 1 -- 172.21.15.47:0/3436025843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 msgr2=0x555e7daea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2da9faf640 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 0x555e7daea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2da9faf640 1 -- 172.21.15.47:0/3436025843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e7d753210 con 0x555e7dadfa30 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2da9faf640 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dadfa30 0x555e7dadfe10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c015de0 tx=0x7f2d9c012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209f2302dced59e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2da9faf640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2d900a3dc0 0x7f2d900a6280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c012c60 tx=0x7f2d9c01d000 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2d8f7fe640 1 -- 172.21.15.47:0/3436025843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00cb60 con 0x555e7dadfa30 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2d8f7fe640 1 -- 172.21.15.47:0/3436025843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d9c00cd00 con 0x555e7dadfa30 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.580+0000 7f2d8f7fe640 1 -- 172.21.15.47:0/3436025843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c005780 con 0x555e7dadfa30 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 -- 172.21.15.47:0/3436025843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dadfa30 msgr2=0x555e7dadfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dadfa30 0x555e7dadfe10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c015de0 tx=0x7f2d9c012760 comp rx=0 tx=0).stop 2024-04-25T21:05:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 -- 172.21.15.47:0/3436025843 shutdown_connections 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 0x555e7daed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 0x555e7daea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/3436025843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dadfa30 0x555e7dadfe10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 -- 172.21.15.47:0/3436025843 >> 172.21.15.47:0/3436025843 conn(0x555e7dadbf40 msgr2=0x555e7dadc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 -- 172.21.15.47:0/3436025843 shutdown_connections 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 -- 172.21.15.47:0/3436025843 wait complete. 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 -- start start 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.581+0000 7f2dae7fd740 1 Processor -- start 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dadfa30 0x555e7daea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 0x555e7dae78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555e7daed7e0 con 0x555e7daeac10 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555e7d9db0b0 con 0x555e7dae0350 2024-04-25T21:05:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555e7daed960 con 0x555e7dadfa30 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2dae7fd740 10 client.0 _ll_register_callbacks cb 0x555e7d9c8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47864/0 (socket says 172.21.15.47:47864) 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 -- 172.21.15.47:0/1940594417 learned_addr learned my addr 172.21.15.47:0/1940594417 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da8fad640 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 0x555e7dae78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da97ae640 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dadfa30 0x555e7daea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 -- 172.21.15.47:0/1940594417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dadfa30 msgr2=0x555e7daea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dadfa30 0x555e7daea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 -- 172.21.15.47:0/1940594417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 msgr2=0x555e7dae78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 0x555e7dae78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 -- 172.21.15.47:0/1940594417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e7d81f620 con 0x555e7daeac10 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2da9faf640 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae7ee0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009740 tx=0x7f2d9c015990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b97cbcd872d6ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.582+0000 7f2d8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2d8cff9640 1 -- 172.21.15.47:0/1940594417 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c005480 con 0x555e7daeac10 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2d8cff9640 1 -- 172.21.15.47:0/1940594417 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f2d9c005620 con 0x555e7daeac10 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 1 -- 172.21.15.47:0/1940594417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9c00cd00 con 0x555e7daeac10 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2d8cff9640 1 -- 172.21.15.47:0/1940594417 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2d9c005820 con 0x555e7daeac10 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 10 client.15873 fetch_fsmap learned FSMap version 35 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 1 -- 172.21.15.47:0/1940594417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555e7daf97b0 con 0x555e7daeac10 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2d8cff9640 1 -- 172.21.15.47:0/1940594417 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d9c00cd00 con 0x555e7daeac10 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2d8cff9640 1 -- 172.21.15.47:0/1940594417 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7f2d9c01e030 con 0x555e7daeac10 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 10 client.15873 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 20 client.15873 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x555e7dafac70 0x555e7dafd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 4 client.15873 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:05:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.583+0000 7f2dae7fd740 1 -- 172.21.15.47:0/1940594417 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e7da4e390 con 0x555e7dafac70 2024-04-25T21:05:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.584+0000 7f2da97ae640 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x555e7dafac70 0x555e7dafd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.584+0000 7f2d8cff9640 10 client.15873 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.584+0000 7f2da97ae640 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x555e7dafac70 0x555e7dafd060 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde5dbe7a3fc6b1b server_cookie=56467599fe8d2b9c in_seq=0 out_seq=0 2024-04-25T21:05:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2d8cff9640 1 -- 172.21.15.47:0/1940594417 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e7da4e390 con 0x555e7dafac70 2024-04-25T21:05:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2d8cff9640 10 client.15873 handle_command_reply: tid=0 2024-04-25T21:05:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 client.15873 shutdown 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 -- 172.21.15.47:0/1940594417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 msgr2=0x555e7dae7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae7ee0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009740 tx=0x7f2d9c015990 comp rx=0 tx=0).stop 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 -- 172.21.15.47:0/1940594417 shutdown_connections 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x555e7dafac70 0x555e7dafd060 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae7ee0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dae0350 0x555e7dae78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2d8cff9640 0 client.15873 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/1940594417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dadfa30 0x555e7daea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 -- 172.21.15.47:0/1940594417 >> 172.21.15.47:0/1940594417 conn(0x555e7dadbf40 msgr2=0x555e7dad6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 -- 172.21.15.47:0/1940594417 shutdown_connections 2024-04-25T21:05:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 1 -- 172.21.15.47:0/1940594417 wait complete. 2024-04-25T21:05:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.585+0000 7f2dae7fd740 20 client.15873 trim_cache size 0 max 16384 2024-04-25T21:05:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2dae7fd740 1 Processor -- start 2024-04-25T21:05:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2dae7fd740 1 -- start start 2024-04-25T21:05:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7dae1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dae0350 0x555e7daea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41ae1b0 con 0x555e7daeac10 2024-04-25T21:05:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41aba50 con 0x555e7dadfa30 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41a98e0 con 0x555e7dae0350 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2da9faf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2da9faf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47870/0 (socket says 172.21.15.47:47870) 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.600+0000 7f2da9faf640 1 -- 172.21.15.47:0/276366448 learned_addr learned my addr 172.21.15.47:0/276366448 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da97ae640 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7dae1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da8fad640 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dae0350 0x555e7daea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da9faf640 1 -- 172.21.15.47:0/276366448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dae0350 msgr2=0x555e7daea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da9faf640 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dae0350 0x555e7daea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da9faf640 1 -- 172.21.15.47:0/276366448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 msgr2=0x555e7dae1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da9faf640 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7dae1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da9faf640 1 -- 172.21.15.47:0/276366448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e7dae84e0 con 0x555e7daeac10 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da8fad640 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dae0350 0x555e7daea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2da9faf640 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae2c40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000f80 tx=0x7f2d9c012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7f57c1f0ba73a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2d8effd640 1 -- 172.21.15.47:0/276366448 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c040070 con 0x555e7daeac10 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2d8effd640 1 -- 172.21.15.47:0/276366448 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d9c005bd0 con 0x555e7daeac10 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.601+0000 7f2d8effd640 1 -- 172.21.15.47:0/276366448 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c03c8d0 con 0x555e7daeac10 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- 172.21.15.47:0/276366448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 msgr2=0x555e7dae2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae2c40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000f80 tx=0x7f2d9c012760 comp rx=0 tx=0).stop 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- 172.21.15.47:0/276366448 shutdown_connections 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7daeac10 0x555e7dae2c40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7dae0350 0x555e7daea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/276366448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7dae1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- 172.21.15.47:0/276366448 >> 172.21.15.47:0/276366448 conn(0x555e7dadbf40 msgr2=0x555e7dae2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- 172.21.15.47:0/276366448 shutdown_connections 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- 172.21.15.47:0/276366448 wait complete. 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- start start 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 Processor -- start 2024-04-25T21:05:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7daf2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dae0350 0x555e7daee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 0x555e7daee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41a9f30 con 0x555e7dae0350 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41ab380 con 0x555e7dadfa30 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41aa5e0 con 0x555e7daeac10 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.602+0000 7f2dae7fd740 10 client.0 _ll_register_callbacks cb 0x7f2d9c015be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dae0350 0x555e7daee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dae0350 0x555e7daee3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47882/0 (socket says 172.21.15.47:47882) 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 -- 172.21.15.47:0/4185100845 learned_addr learned my addr 172.21.15.47:0/4185100845 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da97ae640 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7daf2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da9faf640 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 0x555e7daee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 -- 172.21.15.47:0/4185100845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 msgr2=0x555e7daee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 0x555e7daee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 -- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 msgr2=0x555e7daf2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7daf2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 -- 172.21.15.47:0/4185100845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e7daef140 con 0x555e7dae0350 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da97ae640 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7daf2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2da8fad640 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dae0350 0x555e7daee3b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005fd0 tx=0x7f2d94013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2559934ca2456339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2d8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2d8cff9640 1 -- 172.21.15.47:0/4185100845 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94005020 con 0x555e7dae0350 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2d8cff9640 1 -- 172.21.15.47:0/4185100845 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f2d940051c0 con 0x555e7dae0350 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2dae7fd740 1 -- 172.21.15.47:0/4185100845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9c005bd0 con 0x555e7dae0350 2024-04-25T21:05:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2d8cff9640 1 -- 172.21.15.47:0/4185100845 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2d940053c0 con 0x555e7dae0350 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2dae7fd740 10 client.15879 fetch_fsmap learned FSMap version 35 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.603+0000 7f2dae7fd740 1 -- 172.21.15.47:0/4185100845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555e7db60f10 con 0x555e7dae0350 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2d8cff9640 1 -- 172.21.15.47:0/4185100845 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d9c005bd0 con 0x555e7dae0350 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2d8cff9640 1 -- 172.21.15.47:0/4185100845 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7f2d9400f040 con 0x555e7dae0350 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2dae7fd740 10 client.15879 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2dae7fd740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2dae7fd740 20 client.15879 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x555e7db61740 0x555e7db63b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2dae7fd740 4 client.15879 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2dae7fd740 1 -- 172.21.15.47:0/4185100845 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e7da48200 con 0x555e7db61740 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.604+0000 7f2da97ae640 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x555e7db61740 0x555e7db63b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.605+0000 7f2d8cff9640 10 client.15879 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.605+0000 7f2da97ae640 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x555e7db61740 0x555e7db63b60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9820e2344d44a844 server_cookie=c7326ae061486b8 in_seq=0 out_seq=0 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2d8cff9640 1 -- 172.21.15.47:0/4185100845 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e7da48200 con 0x555e7db61740 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2d8cff9640 10 client.15879 handle_command_reply: tid=0 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 client.15879 shutdown 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 -- 172.21.15.47:0/4185100845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dae0350 msgr2=0x555e7daee3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dae0350 0x555e7daee3b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005fd0 tx=0x7f2d94013830 comp rx=0 tx=0).stop 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 -- 172.21.15.47:0/4185100845 shutdown_connections 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x555e7db61740 0x555e7db63b60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e7daeac10 0x555e7daee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2d8cff9640 0 client.15879 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7dae0350 0x555e7daee3b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 --2- 172.21.15.47:0/4185100845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e7dadfa30 0x555e7daf2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 -- 172.21.15.47:0/4185100845 >> 172.21.15.47:0/4185100845 conn(0x555e7dadbf40 msgr2=0x555e7dae1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 -- 172.21.15.47:0/4185100845 shutdown_connections 2024-04-25T21:05:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 1 -- 172.21.15.47:0/4185100845 wait complete. 2024-04-25T21:05:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.616+0000 7f2dae7fd740 20 client.15879 trim_cache size 0 max 16384 2024-04-25T21:05:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-25T21:05:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-25T21:05:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-25T21:05:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-25T21:05:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 -- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2d900a3dc0 msgr2=0x7f2d900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2d900a3dc0 0x7f2d900a6280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c012c60 tx=0x7f2d9c01d000 comp rx=0 tx=0).stop 2024-04-25T21:05:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 -- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 msgr2=0x7f2da4198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 0x7f2da4198700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2da0005940 tx=0x7f2da00042f0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 -- 172.21.15.47:0/2458338992 shutdown_connections 2024-04-25T21:05:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2da410c5d0 0x7f2da419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f2d900a3dc0 0x7f2d900a6280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da4103b20 0x7f2da4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 --2- 172.21.15.47:0/2458338992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4103200 0x7f2da4198700 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 -- 172.21.15.47:0/2458338992 >> 172.21.15.47:0/2458338992 conn(0x7f2da40771d0 msgr2=0x7f2da40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 -- 172.21.15.47:0/2458338992 shutdown_connections 2024-04-25T21:05:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-25T21:05:39.617+0000 7f2daba12640 1 -- 172.21.15.47:0/2458338992 wait complete. 2024-04-25T21:05:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.966+0000 7ff48616b700 1 -- 172.21.15.47:0/3938145710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 msgr2=0x7ff480101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.966+0000 7ff48616b700 1 --2- 172.21.15.47:0/3938145710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 0x7ff480101db0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff468009990 tx=0x7ff468009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.966+0000 7ff48616b700 1 -- 172.21.15.47:0/3938145710 shutdown_connections 2024-04-25T21:05:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.966+0000 7ff48616b700 1 --2- 172.21.15.47:0/3938145710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4801022f0 0x7ff48010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.966+0000 7ff48616b700 1 --2- 172.21.15.47:0/3938145710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff480107980 0x7ff480107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.966+0000 7ff48616b700 1 --2- 172.21.15.47:0/3938145710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 0x7ff480101db0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.966+0000 7ff48616b700 1 -- 172.21.15.47:0/3938145710 >> 172.21.15.47:0/3938145710 conn(0x7ff4800788b0 msgr2=0x7ff480078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.967+0000 7ff48616b700 1 -- 172.21.15.47:0/3938145710 shutdown_connections 2024-04-25T21:05:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.967+0000 7ff48616b700 1 -- 172.21.15.47:0/3938145710 wait complete. 2024-04-25T21:05:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.967+0000 7ff48616b700 1 Processor -- start 2024-04-25T21:05:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.967+0000 7ff48616b700 1 -- start start 2024-04-25T21:05:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.967+0000 7ff48616b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 0x7ff48019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff48616b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4801022f0 0x7ff48019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff48616b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff480107980 0x7ff48019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff48616b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff480111ad0 con 0x7ff480101930 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff48616b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff480111950 con 0x7ff480107980 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff48616b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff480111c50 con 0x7ff4801022f0 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 0x7ff48019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 0x7ff48019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47896/0 (socket says 172.21.15.47:47896) 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 -- 172.21.15.47:0/3399136140 learned_addr learned my addr 172.21.15.47:0/3399136140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47ffff700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff480107980 0x7ff48019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff476ffd700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4801022f0 0x7ff48019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 -- 172.21.15.47:0/3399136140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4801022f0 msgr2=0x7ff48019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4801022f0 0x7ff48019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 -- 172.21.15.47:0/3399136140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff480107980 msgr2=0x7ff48019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff480107980 0x7ff48019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 -- 172.21.15.47:0/3399136140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4801a0360 con 0x7ff480101930 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47f7fe700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 0x7ff48019b3a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff47000e9a0 tx=0x7ff47000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47d7fa700 1 -- 172.21.15.47:0/3399136140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47000ca10 con 0x7ff480101930 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47d7fa700 1 -- 172.21.15.47:0/3399136140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff47000cbb0 con 0x7ff480101930 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4801a0650 con 0x7ff480101930 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff47d7fa700 1 -- 172.21.15.47:0/3399136140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470018680 con 0x7ff480101930 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.968+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4801a7ed0 con 0x7ff480101930 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.969+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff480069730 con 0x7ff480101930 2024-04-25T21:05:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.971+0000 7ff47d7fa700 1 -- 172.21.15.47:0/3399136140 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff470010430 con 0x7ff480101930 2024-04-25T21:05:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.972+0000 7ff47d7fa700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff46c0a3db0 0x7ff46c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.972+0000 7ff476ffd700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff46c0a3db0 0x7ff46c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.972+0000 7ff47d7fa700 1 -- 172.21.15.47:0/3399136140 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7ff4700d0bc0 con 0x7ff480101930 2024-04-25T21:05:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.972+0000 7ff476ffd700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff46c0a3db0 0x7ff46c0a6270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff480078cd0 tx=0x7ff46800b280 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:39.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:39.976+0000 7ff47d7fa700 1 -- 172.21.15.47:0/3399136140 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff47009d270 con 0x7ff480101930 2024-04-25T21:05:40.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.232+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff48019c6b0 con 0x7ff480101930 2024-04-25T21:05:40.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.233+0000 7ff47d7fa700 1 -- 172.21.15.47:0/3399136140 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11854 (secure 0 0 0) 0x7ff47009cb70 con 0x7ff480101930 2024-04-25T21:05:40.233 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:05:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T21:05:32.309615+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24488,24467]},"id":1}]} 2024-04-25T21:05:40.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff46c0a3db0 msgr2=0x7ff46c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:40.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff46c0a3db0 0x7ff46c0a6270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff480078cd0 tx=0x7ff46800b280 comp rx=0 tx=0).stop 2024-04-25T21:05:40.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 msgr2=0x7ff48019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:40.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 0x7ff48019b3a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff47000e9a0 tx=0x7ff47000ecb0 comp rx=0 tx=0).stop 2024-04-25T21:05:40.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 shutdown_connections 2024-04-25T21:05:40.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7ff46c0a3db0 0x7ff46c0a6270 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:40.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff480101930 0x7ff48019b3a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:40.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff480107980 0x7ff48019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:40.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 --2- 172.21.15.47:0/3399136140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4801022f0 0x7ff48019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:40.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 >> 172.21.15.47:0/3399136140 conn(0x7ff4800788b0 msgr2=0x7ff48006b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:40.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 shutdown_connections 2024-04-25T21:05:40.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:40.235+0000 7ff48616b700 1 -- 172.21.15.47:0/3399136140 wait complete. 2024-04-25T21:05:40.236 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2024-04-25T21:05:40.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:40 smithi186 ceph-mon[32529]: pgmap v173: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-04-25T21:05:40.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:40 smithi047 ceph-mon[29195]: pgmap v173: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-04-25T21:05:40.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:40 smithi092 ceph-mon[32524]: pgmap v173: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-04-25T21:05:40.869 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --cancel --all' 2024-04-25T21:05:40.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24488 quiesce db --cancel --all 2024-04-25T21:05:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:41.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:41 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3399136140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:05:41.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:41 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3399136140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:05:41.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:41 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3399136140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:05:42.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:42 smithi047 ceph-mon[29195]: pgmap v174: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-04-25T21:05:42.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:42 smithi186 ceph-mon[32529]: pgmap v174: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-04-25T21:05:42.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:42 smithi092 ceph-mon[32524]: pgmap v174: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-04-25T21:05:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 -- 172.21.15.47:0/1689235973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 msgr2=0x7f5cc0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1689235973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 0x7f5cc0102f40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009990 tx=0x7f5cb4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 -- 172.21.15.47:0/1689235973 shutdown_connections 2024-04-25T21:05:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1689235973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cc0103480 0x7f5cc010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1689235973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc0108a70 0x7f5cc0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1689235973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 0x7f5cc0102f40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 -- 172.21.15.47:0/1689235973 >> 172.21.15.47:0/1689235973 conn(0x7f5cc0079bb0 msgr2=0x7f5cc0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 -- 172.21.15.47:0/1689235973 shutdown_connections 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 -- 172.21.15.47:0/1689235973 wait complete. 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.921+0000 7f5cc61da700 1 Processor -- start 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cc61da700 1 -- start start 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cc61da700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 0x7f5cc01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cc61da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc0103480 0x7f5cc0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cc61da700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cc0108a70 0x7f5cc019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cc61da700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0112c20 con 0x7f5cc0102ac0 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cc61da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0112aa0 con 0x7f5cc0103480 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cc61da700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0112da0 con 0x7f5cc0108a70 2024-04-25T21:05:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 0x7f5cc01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 0x7f5cc01990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47924/0 (socket says 172.21.15.47:47924) 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/1759296479 learned_addr learned my addr 172.21.15.47:0/1759296479 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbffff700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cc0108a70 0x7f5cc019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc0103480 0x7f5cc0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/1759296479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cc0108a70 msgr2=0x7f5cc019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cc0108a70 0x7f5cc019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/1759296479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc0103480 msgr2=0x7f5cc0199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc0103480 0x7f5cc0199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/1759296479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc019e0b0 con 0x7f5cc0102ac0 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbffff700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cc0108a70 0x7f5cc019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.922+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc0103480 0x7f5cc0199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.923+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 0x7f5cc01990f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000e9a0 tx=0x7f5cb000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.923+0000 7f5cbcff9700 1 -- 172.21.15.47:0/1759296479 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000cbd0 con 0x7f5cc0102ac0 2024-04-25T21:05:42.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.923+0000 7f5cbcff9700 1 -- 172.21.15.47:0/1759296479 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb000cd70 con 0x7f5cc0102ac0 2024-04-25T21:05:42.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.923+0000 7f5cc61da700 1 -- 172.21.15.47:0/1759296479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc019e3a0 con 0x7f5cc0102ac0 2024-04-25T21:05:42.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.923+0000 7f5cbcff9700 1 -- 172.21.15.47:0/1759296479 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0018680 con 0x7f5cc0102ac0 2024-04-25T21:05:42.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.923+0000 7f5cc61da700 1 -- 172.21.15.47:0/1759296479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc01a5c20 con 0x7f5cc0102ac0 2024-04-25T21:05:42.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.925+0000 7f5cbcff9700 1 -- 172.21.15.47:0/1759296479 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5cb0010970 con 0x7f5cc0102ac0 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.926+0000 7f5cbcff9700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5ca80a3db0 0x7f5ca80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.926+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5ca80a3db0 0x7f5ca80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.926+0000 7f5cbcff9700 1 -- 172.21.15.47:0/1759296479 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f5cb00d0ae0 con 0x7f5cc0102ac0 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.926+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5ca80a3db0 0x7f5ca80a6270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400b2e0 tx=0x7f5cb4005e80 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.926+0000 7f5cd0a63b80 1 Processor -- start 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cd0a63b80 1 -- start start 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49bed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 0x5617f49ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5617f4911da0 con 0x5617f49be960 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5617f48b89d0 con 0x5617f49bf280 2024-04-25T21:05:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5617f490dd60 con 0x5617f49ce560 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49bed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47936/0 (socket says 172.21.15.47:47936) 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/2471317063 learned_addr learned my addr 172.21.15.47:0/2471317063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbffff700 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 0x5617f49ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/2471317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 msgr2=0x5617f49d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/2471317063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 msgr2=0x5617f49ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 0x5617f49ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/2471317063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f485f940 con 0x5617f49be960 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbffff700 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.927+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49bed40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00a51d0 tx=0x7f5cb009d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61e7189b8ea3e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5ca5ffb700 1 -- 172.21.15.47:0/2471317063 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb009ad90 con 0x5617f49be960 2024-04-25T21:05:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5ca5ffb700 1 -- 172.21.15.47:0/2471317063 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb009ccf0 con 0x5617f49be960 2024-04-25T21:05:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5ca5ffb700 1 -- 172.21.15.47:0/2471317063 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0098a20 con 0x5617f49be960 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2471317063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 msgr2=0x5617f49bed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49bed40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00a51d0 tx=0x7f5cb009d2e0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2471317063 shutdown_connections 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49bed40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/2471317063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 0x5617f49ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2471317063 >> 172.21.15.47:0/2471317063 conn(0x5617f48ec700 msgr2=0x5617f49bae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2471317063 shutdown_connections 2024-04-25T21:05:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.928+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2471317063 wait complete. 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 1 -- start start 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 1 Processor -- start 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49bf280 0x5617f49cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49ce560 0x5617f49cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5617f49d0ef0 con 0x5617f49be960 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5617f48b0c40 con 0x5617f49ce560 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5617f49d1070 con 0x5617f49bf280 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.929+0000 7f5cd0a63b80 10 client.0 _ll_register_callbacks cb 0x5617f48aa9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47950/0 (socket says 172.21.15.47:47950) 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/119665779 learned_addr learned my addr 172.21.15.47:0/119665779 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49bf280 0x5617f49cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbffff700 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49ce560 0x5617f49cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/119665779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49bf280 msgr2=0x5617f49cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49bf280 0x5617f49cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/119665779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49ce560 msgr2=0x5617f49cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49ce560 0x5617f49cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/119665779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f48c92e0 con 0x5617f49be960 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49cdd70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0098ef0 tx=0x7f5cb0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bdc20943270654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5c9effd700 1 -- 172.21.15.47:0/119665779 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb009c0e0 con 0x5617f49be960 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5c9effd700 1 -- 172.21.15.47:0/119665779 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f5cb00987e0 con 0x5617f49be960 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/119665779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb009ccf0 con 0x5617f49be960 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5c9effd700 1 -- 172.21.15.47:0/119665779 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5cb00bd0c0 con 0x5617f49be960 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cd0a63b80 10 client.15897 fetch_fsmap learned FSMap version 36 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.930+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/119665779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5617f49db800 con 0x5617f49be960 2024-04-25T21:05:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5c9effd700 1 -- 172.21.15.47:0/119665779 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb009ccf0 con 0x5617f49be960 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5c9effd700 1 -- 172.21.15.47:0/119665779 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f5cb00b9070 con 0x5617f49be960 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5cd0a63b80 10 client.15897 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5cd0a63b80 10 client.15897 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5cd0a63b80 20 client.15897 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5617f49ddcd0 0x5617f49e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5cd0a63b80 4 client.15897 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/119665779 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617f491f590 con 0x5617f49ddcd0 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.931+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5617f49ddcd0 0x5617f49e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.932+0000 7f5c9effd700 10 client.15897 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.932+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5617f49ddcd0 0x5617f49e00c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8be0898855f74a server_cookie=4bbec25c029a96e3 in_seq=0 out_seq=0 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.932+0000 7f5c9effd700 1 -- 172.21.15.47:0/119665779 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5617f491f590 con 0x5617f49ddcd0 2024-04-25T21:05:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.932+0000 7f5c9effd700 10 client.15897 handle_command_reply: tid=0 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 client.15897 shutdown 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/119665779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 msgr2=0x5617f49cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49cdd70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0098ef0 tx=0x7f5cb0009790 comp rx=0 tx=0).stop 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/119665779 shutdown_connections 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5617f49ddcd0 0x5617f49e00c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49cdd70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49ce560 0x5617f49cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5c9effd700 0 client.15897 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/119665779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49bf280 0x5617f49cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/119665779 >> 172.21.15.47:0/119665779 conn(0x5617f48ec700 msgr2=0x5617f49baa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/119665779 shutdown_connections 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/119665779 wait complete. 2024-04-25T21:05:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.933+0000 7f5cd0a63b80 20 client.15897 trim_cache size 0 max 16384 2024-04-25T21:05:42.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.952+0000 7f5cd0a63b80 1 Processor -- start 2024-04-25T21:05:42.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.952+0000 7f5cd0a63b80 1 -- start start 2024-04-25T21:05:42.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 0x5617f49baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01ad9c0 con 0x5617f49be960 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01ab1f0 con 0x5617f49bf280 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01a9060 con 0x5617f49ce560 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47952/0 (socket says 172.21.15.47:47952) 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/2593682195 learned_addr learned my addr 172.21.15.47:0/2593682195 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 0x5617f49baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbffff700 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/2593682195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 msgr2=0x5617f49e30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49e30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/2593682195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 msgr2=0x5617f49baf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 0x5617f49baf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/2593682195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f49ce0e0 con 0x5617f49be960 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49ccf10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00ce840 tx=0x7f5cb00bd050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9f59c77b026063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.953+0000 7f5ca57fa700 1 -- 172.21.15.47:0/2593682195 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb009ccf0 con 0x5617f49be960 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5ca57fa700 1 -- 172.21.15.47:0/2593682195 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb009b950 con 0x5617f49be960 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5ca57fa700 1 -- 172.21.15.47:0/2593682195 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0010430 con 0x5617f49be960 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2593682195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 msgr2=0x5617f49ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49ccf10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00ce840 tx=0x7f5cb00bd050 comp rx=0 tx=0).stop 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2593682195 shutdown_connections 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49be960 0x5617f49ccf10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49e30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/2593682195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49bf280 0x5617f49baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2593682195 >> 172.21.15.47:0/2593682195 conn(0x5617f48ec700 msgr2=0x5617f49ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2593682195 shutdown_connections 2024-04-25T21:05:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/2593682195 wait complete. 2024-04-25T21:05:42.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.954+0000 7f5cd0a63b80 1 -- start start 2024-04-25T21:05:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cd0a63b80 1 Processor -- start 2024-04-25T21:05:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49be960 0x5617f49e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49bf280 0x5617f49df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cd0a63b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01a96b0 con 0x5617f49bf280 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01aab20 con 0x5617f49be960 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cd0a63b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01a9d80 con 0x5617f49ce560 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cd0a63b80 10 client.0 _ll_register_callbacks cb 0x5617f48aa9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49bf280 0x5617f49df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49be960 0x5617f49e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49bf280 0x5617f49df630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47960/0 (socket says 172.21.15.47:47960) 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbeffd700 1 -- 172.21.15.47:0/146979746 learned_addr learned my addr 172.21.15.47:0/146979746 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbffff700 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/146979746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 msgr2=0x5617f49dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49dfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/146979746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49bf280 msgr2=0x5617f49df630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:05:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.955+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49bf280 0x5617f49df630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5cbf7fe700 1 -- 172.21.15.47:0/146979746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f49e01b0 con 0x5617f49be960 2024-04-25T21:05:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49bf280 0x5617f49df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5cbf7fe700 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49be960 0x5617f49e32a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5cb009ad40 tx=0x7f5cb000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccabc02e8d00bb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-25T21:05:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5c9effd700 1 -- 172.21.15.47:0/146979746 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb009c0e0 con 0x5617f49be960 2024-04-25T21:05:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/146979746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb009b950 con 0x5617f49be960 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5c9effd700 1 -- 172.21.15.47:0/146979746 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8506+0+0 (secure 0 0 0) 0x7f5cb0010430 con 0x5617f49be960 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5c9effd700 1 -- 172.21.15.47:0/146979746 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5cb009ccf0 con 0x5617f49be960 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5cd0a63b80 10 client.25514 fetch_fsmap learned FSMap version 36 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/146979746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5617f49e01b0 con 0x5617f49be960 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5c9effd700 1 -- 172.21.15.47:0/146979746 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb009b950 con 0x5617f49be960 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.956+0000 7f5c9effd700 1 -- 172.21.15.47:0/146979746 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f5cb00b9070 con 0x5617f49be960 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cd0a63b80 10 client.25514 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cd0a63b80 10 client.25514 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cd0a63b80 20 client.25514 populate_metadata read hostname 'smithi047' 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5617f4a45020 0x5617f4a47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cd0a63b80 4 client.25514 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/146979746 --> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5617f4919630 con 0x5617f4a45020 2024-04-25T21:05:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5617f4a45020 0x5617f4a47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cbeffd700 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5617f4a45020 0x5617f4a47440 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a051b658295b8e server_cookie=c9b7ac714c590b10 in_seq=0 out_seq=0 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5c9effd700 10 client.25514 ms_handle_connect on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5c9effd700 1 -- 172.21.15.47:0/146979746 <== mds.0 v2:172.21.15.92:6838/941998037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x5617f4919630 con 0x5617f4a45020 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5c9effd700 10 client.25514 handle_command_reply: tid=0 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.957+0000 7f5cd0a63b80 1 client.25514 shutdown 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/146979746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49be960 msgr2=0x5617f49e32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49be960 0x5617f49e32a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5cb009ad40 tx=0x7f5cb000cf90 comp rx=0 tx=0).stop 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/146979746 shutdown_connections 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.92:6838/941998037,v1:172.21.15.92:6839/941998037] conn(0x5617f4a45020 0x5617f4a47440 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f49ce560 0x5617f49dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617f49be960 0x5617f49e32a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 --2- 172.21.15.47:0/146979746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617f49bf280 0x5617f49df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5c9effd700 0 client.25514 ms_handle_reset on v2:172.21.15.92:6838/941998037 2024-04-25T21:05:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/146979746 >> 172.21.15.47:0/146979746 conn(0x5617f48ec700 msgr2=0x5617f49ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/146979746 shutdown_connections 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 1 -- 172.21.15.47:0/146979746 wait complete. 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cd0a63b80 20 client.25514 trim_cache size 0 max 16384 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 36, 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stdout: "leader": 24488, 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stdout: "set_version": 6, 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stdout: "sets": {} 2024-04-25T21:05:42.968 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:05:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 -- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5ca80a3db0 msgr2=0x7f5ca80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5ca80a3db0 0x7f5ca80a6270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400b2e0 tx=0x7f5cb4005e80 comp rx=0 tx=0).stop 2024-04-25T21:05:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 -- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 msgr2=0x7f5cc01990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 0x7f5cc01990f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000e9a0 tx=0x7f5cb000ed60 comp rx=0 tx=0).stop 2024-04-25T21:05:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 -- 172.21.15.47:0/1759296479 shutdown_connections 2024-04-25T21:05:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f5ca80a3db0 0x7f5ca80a6270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cc0102ac0 0x7f5cc01990f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cc0108a70 0x7f5cc019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 --2- 172.21.15.47:0/1759296479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc0103480 0x7f5cc0199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.958+0000 7f5cc61da700 1 -- 172.21.15.47:0/1759296479 >> 172.21.15.47:0/1759296479 conn(0x7f5cc0079bb0 msgr2=0x7f5cc01017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.959+0000 7f5cc61da700 1 -- 172.21.15.47:0/1759296479 shutdown_connections 2024-04-25T21:05:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:05:42.959+0000 7f5cc61da700 1 -- 172.21.15.47:0/1759296479 wait complete. 2024-04-25T21:05:43.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:43 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:43.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:43 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:43.577 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:43 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:43.612 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks/quiescer.py", line 418, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks/quiescer.py", line 377, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error quiescing set '9ebddfb8': 110 (ETIMEDOUT) 2024-04-25T21:05:43.613 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T21:05:43.623 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T21:05:43.632 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T21:05:43.648 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T21:05:43.648 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T21:05:43.648 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T21:05:43.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T21:05:43.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T21:05:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:44 smithi186 ceph-mon[32529]: pgmap v175: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 266 op/s 2024-04-25T21:05:44.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:44 smithi092 ceph-mon[32524]: pgmap v175: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 266 op/s 2024-04-25T21:05:44.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:44 smithi047 ceph-mon[29195]: pgmap v175: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 266 op/s 2024-04-25T21:05:46.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:46 smithi186 ceph-mon[32529]: pgmap v176: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-04-25T21:05:46.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:46 smithi092 ceph-mon[32524]: pgmap v176: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-04-25T21:05:46.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:46 smithi047 ceph-mon[29195]: pgmap v176: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-04-25T21:05:48.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:48 smithi092 ceph-mon[32524]: pgmap v177: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 455 KiB/s wr, 116 op/s 2024-04-25T21:05:48.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:48 smithi047 ceph-mon[29195]: pgmap v177: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 455 KiB/s wr, 116 op/s 2024-04-25T21:05:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:48 smithi186 ceph-mon[32529]: pgmap v177: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 455 KiB/s wr, 116 op/s 2024-04-25T21:05:50.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:50 smithi092 ceph-mon[32524]: pgmap v178: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 455 KiB/s wr, 116 op/s 2024-04-25T21:05:50.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:50 smithi047 ceph-mon[29195]: pgmap v178: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 455 KiB/s wr, 116 op/s 2024-04-25T21:05:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:50 smithi186 ceph-mon[32529]: pgmap v178: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 455 KiB/s wr, 116 op/s 2024-04-25T21:05:52.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:52 smithi092 ceph-mon[32524]: pgmap v179: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 458 KiB/s wr, 120 op/s 2024-04-25T21:05:52.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:52 smithi047 ceph-mon[29195]: pgmap v179: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 458 KiB/s wr, 120 op/s 2024-04-25T21:05:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:52 smithi186 ceph-mon[32529]: pgmap v179: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 458 KiB/s wr, 120 op/s 2024-04-25T21:05:54.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:54 smithi092 ceph-mon[32524]: pgmap v180: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-04-25T21:05:54.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:54 smithi047 ceph-mon[29195]: pgmap v180: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-04-25T21:05:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:54 smithi186 ceph-mon[32529]: pgmap v180: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-04-25T21:05:56.522 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:56 smithi186 ceph-mon[32529]: pgmap v181: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:05:56.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:56 smithi092 ceph-mon[32524]: pgmap v181: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:05:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:56 smithi047 ceph-mon[29195]: pgmap v181: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:05:58.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:05:58 smithi092 ceph-mon[32524]: pgmap v182: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:05:58.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:05:58 smithi047 ceph-mon[29195]: pgmap v182: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:05:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:05:58 smithi186 ceph-mon[32529]: pgmap v182: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:06:00.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:00 smithi092 ceph-mon[32524]: pgmap v183: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:06:00.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:00 smithi047 ceph-mon[29195]: pgmap v183: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:06:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:00 smithi186 ceph-mon[32529]: pgmap v183: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:06:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:02 smithi092 ceph-mon[32524]: pgmap v184: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-25T21:06:02.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:02 smithi047 ceph-mon[29195]: pgmap v184: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-25T21:06:02.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:02 smithi186 ceph-mon[32529]: pgmap v184: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-25T21:06:04.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:04 smithi092 ceph-mon[32524]: pgmap v185: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T21:06:04.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:04 smithi047 ceph-mon[29195]: pgmap v185: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T21:06:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:04 smithi186 ceph-mon[32529]: pgmap v185: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T21:06:06.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:06 smithi092 ceph-mon[32524]: pgmap v186: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-25T21:06:06.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:06 smithi047 ceph-mon[29195]: pgmap v186: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-25T21:06:06.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:06 smithi186 ceph-mon[32529]: pgmap v186: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-25T21:06:08.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:08 smithi092 ceph-mon[32524]: pgmap v187: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:08.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:08 smithi047 ceph-mon[29195]: pgmap v187: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:08 smithi186 ceph-mon[32529]: pgmap v187: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:10 smithi092 ceph-mon[32524]: pgmap v188: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:10.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:10 smithi047 ceph-mon[29195]: pgmap v188: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:10.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:10 smithi186 ceph-mon[32529]: pgmap v188: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:11.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:11 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:06:11.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:11 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:06:11.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:11 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:06:11.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:11 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:06:11.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:11 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:06:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:11 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:06:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:11 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:06:11.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:11 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:06:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:11 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:06:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:11 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:06:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:11 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:06:11.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:11 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:06:12.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:12 smithi092 ceph-mon[32524]: pgmap v189: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:06:12.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:12 smithi047 ceph-mon[29195]: pgmap v189: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:06:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:12 smithi186 ceph-mon[32529]: pgmap v189: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:06:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:14 smithi092 ceph-mon[32524]: pgmap v190: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:14.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:14 smithi047 ceph-mon[29195]: pgmap v190: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:14 smithi186 ceph-mon[32529]: pgmap v190: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:06:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:16 smithi092 ceph-mon[32524]: pgmap v191: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:06:16.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:16 smithi047 ceph-mon[29195]: pgmap v191: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:06:16.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:16 smithi186 ceph-mon[32529]: pgmap v191: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T21:06:18.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:18 smithi092 ceph-mon[32524]: pgmap v192: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:06:18.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:18 smithi047 ceph-mon[29195]: pgmap v192: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:06:18.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:18 smithi186 ceph-mon[32529]: pgmap v192: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:06:20.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:20 smithi092 ceph-mon[32524]: pgmap v193: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:06:20.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:20 smithi047 ceph-mon[29195]: pgmap v193: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:06:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:20 smithi186 ceph-mon[32529]: pgmap v193: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:06:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:22 smithi092 ceph-mon[32524]: pgmap v194: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:06:22.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:22 smithi047 ceph-mon[29195]: pgmap v194: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:06:22.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:22 smithi186 ceph-mon[32529]: pgmap v194: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:06:24.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:24 smithi092 ceph-mon[32524]: pgmap v195: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:24.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:24 smithi047 ceph-mon[29195]: pgmap v195: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:24.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:24 smithi186 ceph-mon[32529]: pgmap v195: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:26.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:26 smithi092 ceph-mon[32524]: pgmap v196: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:26.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:26 smithi047 ceph-mon[29195]: pgmap v196: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:26 smithi186 ceph-mon[32529]: pgmap v196: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:28.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:28 smithi092 ceph-mon[32524]: pgmap v197: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:28 smithi047 ceph-mon[29195]: pgmap v197: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:28.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:28 smithi186 ceph-mon[32529]: pgmap v197: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:30.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:30 smithi047 ceph-mon[29195]: pgmap v198: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:30.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:30 smithi186 ceph-mon[32529]: pgmap v198: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:30.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:30 smithi092 ceph-mon[32524]: pgmap v198: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:32.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:32 smithi047 ceph-mon[29195]: pgmap v199: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:32.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:32 smithi186 ceph-mon[32529]: pgmap v199: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:32.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:32 smithi092 ceph-mon[32524]: pgmap v199: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:34.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:34 smithi047 ceph-mon[29195]: pgmap v200: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:34.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:34 smithi186 ceph-mon[32529]: pgmap v200: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:34.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:34 smithi092 ceph-mon[32524]: pgmap v200: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:36.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:36 smithi047 ceph-mon[29195]: pgmap v201: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:36.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:36 smithi186 ceph-mon[32529]: pgmap v201: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:36.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:36 smithi092 ceph-mon[32524]: pgmap v201: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:38.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:38 smithi047 ceph-mon[29195]: pgmap v202: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:38 smithi186 ceph-mon[32529]: pgmap v202: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:38.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:38 smithi092 ceph-mon[32524]: pgmap v202: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:40.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:40 smithi047 ceph-mon[29195]: pgmap v203: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:40.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:40 smithi186 ceph-mon[32529]: pgmap v203: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:40.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:40 smithi092 ceph-mon[32524]: pgmap v203: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:42.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:42 smithi047 ceph-mon[29195]: pgmap v204: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:42.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:42 smithi186 ceph-mon[32529]: pgmap v204: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:42.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:42 smithi092 ceph-mon[32524]: pgmap v204: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:44.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:44 smithi047 ceph-mon[29195]: pgmap v205: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:44.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:44 smithi186 ceph-mon[32529]: pgmap v205: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:44.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:44 smithi092 ceph-mon[32524]: pgmap v205: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:46.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:46 smithi047 ceph-mon[29195]: pgmap v206: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:46.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:46 smithi186 ceph-mon[32529]: pgmap v206: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:46.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:46 smithi092 ceph-mon[32524]: pgmap v206: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:48.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:48 smithi047 ceph-mon[29195]: pgmap v207: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:48.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:48 smithi186 ceph-mon[32529]: pgmap v207: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:48 smithi092 ceph-mon[32524]: pgmap v207: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:50.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:50 smithi047 ceph-mon[29195]: pgmap v208: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:50 smithi186 ceph-mon[32529]: pgmap v208: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:50.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:50 smithi092 ceph-mon[32524]: pgmap v208: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:52.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:52 smithi047 ceph-mon[29195]: pgmap v209: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:52 smithi186 ceph-mon[32529]: pgmap v209: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:52.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:52 smithi092 ceph-mon[32524]: pgmap v209: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:54.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:54 smithi047 ceph-mon[29195]: pgmap v210: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:54 smithi186 ceph-mon[32529]: pgmap v210: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:54 smithi092 ceph-mon[32524]: pgmap v210: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:56 smithi047 ceph-mon[29195]: pgmap v211: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:56 smithi186 ceph-mon[32529]: pgmap v211: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:56 smithi092 ceph-mon[32524]: pgmap v211: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:58.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:06:58 smithi047 ceph-mon[29195]: pgmap v212: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:06:58 smithi186 ceph-mon[32529]: pgmap v212: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:58.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:06:58 smithi092 ceph-mon[32524]: pgmap v212: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:00.584 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:00 smithi092 ceph-mon[32524]: pgmap v213: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:00.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:00 smithi047 ceph-mon[29195]: pgmap v213: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:00 smithi186 ceph-mon[32529]: pgmap v213: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:02.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:02 smithi047 ceph-mon[29195]: pgmap v214: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:02.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:02 smithi186 ceph-mon[32529]: pgmap v214: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:02.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:02 smithi092 ceph-mon[32524]: pgmap v214: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:04.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:04 smithi047 ceph-mon[29195]: pgmap v215: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:04 smithi186 ceph-mon[32529]: pgmap v215: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:04 smithi092 ceph-mon[32524]: pgmap v215: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:06.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:06 smithi047 ceph-mon[29195]: pgmap v216: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:07:06.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:06 smithi186 ceph-mon[32529]: pgmap v216: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:07:06.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:06 smithi092 ceph-mon[32524]: pgmap v216: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:07:08.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:08 smithi047 ceph-mon[29195]: pgmap v217: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:08.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:08 smithi047 ceph-mon[29195]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T21:07:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:08 smithi186 ceph-mon[32529]: pgmap v217: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:08 smithi186 ceph-mon[32529]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T21:07:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:08 smithi092 ceph-mon[32524]: pgmap v217: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:08 smithi092 ceph-mon[32524]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T21:07:10.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:10 smithi047 ceph-mon[29195]: pgmap v219: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:10.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:10 smithi186 ceph-mon[32529]: pgmap v219: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:10.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:10 smithi092 ceph-mon[32524]: pgmap v219: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:11.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:11 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:07:11.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:11 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:07:11.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:11 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:07:12.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:12 smithi047 ceph-mon[29195]: pgmap v220: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:12.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:12 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:12.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:12 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:12.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:12 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:07:12.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:12 smithi047 ceph-mon[29195]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T21:07:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:12 smithi186 ceph-mon[32529]: pgmap v220: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:12 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:12 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:12 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:07:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:12 smithi186 ceph-mon[32529]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T21:07:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:12 smithi092 ceph-mon[32524]: pgmap v220: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:12 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:12 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:12 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:07:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:12 smithi092 ceph-mon[32524]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T21:07:14.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:14 smithi047 ceph-mon[29195]: pgmap v222: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:07:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:14 smithi186 ceph-mon[32529]: pgmap v222: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:07:14.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:14 smithi092 ceph-mon[32524]: pgmap v222: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:07:16.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:16 smithi047 ceph-mon[29195]: pgmap v223: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:07:16.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:16 smithi186 ceph-mon[32529]: pgmap v223: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:07:16.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:16 smithi092 ceph-mon[32524]: pgmap v223: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:07:18.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:18 smithi186 ceph-mon[32529]: pgmap v224: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-04-25T21:07:18.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:18 smithi092 ceph-mon[32524]: pgmap v224: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-04-25T21:07:18.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:18 smithi047 ceph-mon[29195]: pgmap v224: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-04-25T21:07:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:20 smithi186 ceph-mon[32529]: pgmap v225: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:07:20.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:20 smithi092 ceph-mon[32524]: pgmap v225: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:07:20.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:20 smithi047 ceph-mon[29195]: pgmap v225: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:07:22.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:22 smithi186 ceph-mon[32529]: pgmap v226: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:22.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:22 smithi092 ceph-mon[32524]: pgmap v226: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:22.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:22 smithi047 ceph-mon[29195]: pgmap v226: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:24.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:24 smithi186 ceph-mon[32529]: pgmap v227: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:07:24.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:24 smithi092 ceph-mon[32524]: pgmap v227: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:07:24.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:24 smithi047 ceph-mon[29195]: pgmap v227: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:07:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:26 smithi186 ceph-mon[32529]: pgmap v228: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:07:26.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:26 smithi092 ceph-mon[32524]: pgmap v228: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:07:26.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:26 smithi047 ceph-mon[29195]: pgmap v228: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:07:28.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:28 smithi186 ceph-mon[32529]: pgmap v229: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:28.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:28 smithi092 ceph-mon[32524]: pgmap v229: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:28.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:28 smithi047 ceph-mon[29195]: pgmap v229: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:30 smithi186 ceph-mon[32529]: pgmap v230: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:30.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:30 smithi092 ceph-mon[32524]: pgmap v230: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:30.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:30 smithi047 ceph-mon[29195]: pgmap v230: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:32.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:32 smithi186 ceph-mon[32529]: pgmap v231: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:32.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:32 smithi092 ceph-mon[32524]: pgmap v231: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:32.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:32 smithi047 ceph-mon[29195]: pgmap v231: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:34.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:34 smithi186 ceph-mon[32529]: pgmap v232: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:34.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:34 smithi092 ceph-mon[32524]: pgmap v232: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:34.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:34 smithi047 ceph-mon[29195]: pgmap v232: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:36.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:36 smithi186 ceph-mon[32529]: pgmap v233: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:36.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:36 smithi092 ceph-mon[32524]: pgmap v233: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:36.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:36 smithi047 ceph-mon[29195]: pgmap v233: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:38.663 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:38 smithi047 ceph-mon[29195]: pgmap v234: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:38.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:38 smithi186 ceph-mon[32529]: pgmap v234: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:38.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:38 smithi092 ceph-mon[32524]: pgmap v234: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:40.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:40 smithi186 ceph-mon[32529]: pgmap v235: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:40.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:40 smithi092 ceph-mon[32524]: pgmap v235: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:40.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:40 smithi047 ceph-mon[29195]: pgmap v235: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T21:07:42.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:42 smithi186 ceph-mon[32529]: pgmap v236: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:42.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:42 smithi092 ceph-mon[32524]: pgmap v236: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:42.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:42 smithi047 ceph-mon[29195]: pgmap v236: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:44.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:44 smithi186 ceph-mon[32529]: pgmap v237: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:44.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:44 smithi092 ceph-mon[32524]: pgmap v237: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:44.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:44 smithi047 ceph-mon[29195]: pgmap v237: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:46.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:46 smithi186 ceph-mon[32529]: pgmap v238: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:46.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:46 smithi092 ceph-mon[32524]: pgmap v238: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:46.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:46 smithi047 ceph-mon[29195]: pgmap v238: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:48 smithi186 ceph-mon[32529]: pgmap v239: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:48 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:07:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:48 smithi092 ceph-mon[32524]: pgmap v239: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:48 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:07:48.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:48 smithi047 ceph-mon[29195]: pgmap v239: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:48.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:48 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:07:50.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:50 smithi186 ceph-mon[32529]: pgmap v240: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:50.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:50 smithi092 ceph-mon[32524]: pgmap v240: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:50.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:50 smithi047 ceph-mon[29195]: pgmap v240: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:51.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:51 smithi186 ceph-mon[32529]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:07:51.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:51 smithi092 ceph-mon[32524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:07:51.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:51 smithi047 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:07:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:52 smithi186 ceph-mon[32529]: pgmap v241: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:52.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:52 smithi092 ceph-mon[32524]: pgmap v241: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:52.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:52 smithi047 ceph-mon[29195]: pgmap v241: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:54 smithi186 ceph-mon[32529]: pgmap v242: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:54.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:54 smithi092 ceph-mon[32524]: pgmap v242: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:54.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:54 smithi047 ceph-mon[29195]: pgmap v242: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:56 smithi186 ceph-mon[32529]: pgmap v243: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:56.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:56 smithi092 ceph-mon[32524]: pgmap v243: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:56.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:56 smithi047 ceph-mon[29195]: pgmap v243: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:07:58 smithi186 ceph-mon[32529]: pgmap v244: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:58.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:07:58 smithi092 ceph-mon[32524]: pgmap v244: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:58.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:07:58 smithi047 ceph-mon[29195]: pgmap v244: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:00 smithi186 ceph-mon[32529]: pgmap v245: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:00.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:00 smithi092 ceph-mon[32524]: pgmap v245: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:00.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:00 smithi047 ceph-mon[29195]: pgmap v245: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:02.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:02 smithi186 ceph-mon[32529]: pgmap v246: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:02.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:02 smithi092 ceph-mon[32524]: pgmap v246: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:02.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:02 smithi047 ceph-mon[29195]: pgmap v246: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:04 smithi186 ceph-mon[32529]: pgmap v247: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:04 smithi092 ceph-mon[32524]: pgmap v247: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:04.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:04 smithi047 ceph-mon[29195]: pgmap v247: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:06.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:06 smithi186 ceph-mon[32529]: pgmap v248: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:08:06.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:06 smithi092 ceph-mon[32524]: pgmap v248: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:08:06.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:06 smithi047 ceph-mon[29195]: pgmap v248: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:08:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:08 smithi186 ceph-mon[32529]: pgmap v249: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:08:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:08 smithi186 ceph-mon[32529]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:08:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:08 smithi092 ceph-mon[32524]: pgmap v249: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:08:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:08 smithi092 ceph-mon[32524]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:08:08.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:08 smithi047 ceph-mon[29195]: pgmap v249: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:08:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:08 smithi047 ceph-mon[29195]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:08:10.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:10 smithi186 ceph-mon[32529]: pgmap v251: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:08:10.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:10 smithi092 ceph-mon[32524]: pgmap v251: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:08:10.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:10 smithi047 ceph-mon[29195]: pgmap v251: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:08:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:12 smithi186 ceph-mon[32529]: pgmap v252: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:08:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:12 smithi186 ceph-mon[32529]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:08:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:12 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:12.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:12 smithi092 ceph-mon[32524]: pgmap v252: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:08:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:12 smithi092 ceph-mon[32524]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:08:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:12 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:12.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:12 smithi047 ceph-mon[29195]: pgmap v252: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:08:12.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:12 smithi047 ceph-mon[29195]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:08:12.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:12 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:13.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:13 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:13.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:13 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:13.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:13 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:08:13.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:13 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:13.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:13 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:13.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:13 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:08:13.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:13 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:13.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:13 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:13.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:13 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:08:14.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:14 smithi047 ceph-mon[29195]: pgmap v254: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:08:14.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:14 smithi186 ceph-mon[32529]: pgmap v254: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:08:15.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:14 smithi092 ceph-mon[32524]: pgmap v254: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:08:16.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:16 smithi186 ceph-mon[32529]: pgmap v255: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:08:17.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:16 smithi092 ceph-mon[32524]: pgmap v255: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:08:17.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:16 smithi047 ceph-mon[29195]: pgmap v255: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:08:18.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:18 smithi186 ceph-mon[32529]: pgmap v256: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-04-25T21:08:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:18 smithi092 ceph-mon[32524]: pgmap v256: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-04-25T21:08:19.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:18 smithi047 ceph-mon[29195]: pgmap v256: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-04-25T21:08:20.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:20 smithi186 ceph-mon[32529]: pgmap v257: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:08:21.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:20 smithi092 ceph-mon[32524]: pgmap v257: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:08:21.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:20 smithi047 ceph-mon[29195]: pgmap v257: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:08:22.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:22 smithi186 ceph-mon[32529]: pgmap v258: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:08:23.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:22 smithi092 ceph-mon[32524]: pgmap v258: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:08:23.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:22 smithi047 ceph-mon[29195]: pgmap v258: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:08:24.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:24 smithi186 ceph-mon[32529]: pgmap v259: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:08:25.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:24 smithi092 ceph-mon[32524]: pgmap v259: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:08:25.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:24 smithi047 ceph-mon[29195]: pgmap v259: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:08:26.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:26 smithi186 ceph-mon[32529]: pgmap v260: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:08:27.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:26 smithi092 ceph-mon[32524]: pgmap v260: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:08:27.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:26 smithi047 ceph-mon[29195]: pgmap v260: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:08:28.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:28 smithi186 ceph-mon[32529]: pgmap v261: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:29.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:28 smithi092 ceph-mon[32524]: pgmap v261: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:29.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:28 smithi047 ceph-mon[29195]: pgmap v261: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:30.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:30 smithi186 ceph-mon[32529]: pgmap v262: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:31.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:30 smithi092 ceph-mon[32524]: pgmap v262: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:31.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:30 smithi047 ceph-mon[29195]: pgmap v262: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:32.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:32 smithi186 ceph-mon[32529]: pgmap v263: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:33.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:32 smithi092 ceph-mon[32524]: pgmap v263: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:33.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:32 smithi047 ceph-mon[29195]: pgmap v263: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:34.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:34 smithi186 ceph-mon[32529]: pgmap v264: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:35.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:34 smithi092 ceph-mon[32524]: pgmap v264: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:35.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:34 smithi047 ceph-mon[29195]: pgmap v264: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:36 smithi186 ceph-mon[32529]: pgmap v265: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:37.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:36 smithi092 ceph-mon[32524]: pgmap v265: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:37.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:36 smithi047 ceph-mon[29195]: pgmap v265: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:38.913 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:38 smithi047 ceph-mon[29195]: pgmap v266: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:38.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:38 smithi186 ceph-mon[32529]: pgmap v266: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:39.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:38 smithi092 ceph-mon[32524]: pgmap v266: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:40.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:40 smithi186 ceph-mon[32529]: pgmap v267: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:41.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:40 smithi092 ceph-mon[32524]: pgmap v267: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:41.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:40 smithi047 ceph-mon[29195]: pgmap v267: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:42 smithi186 ceph-mon[32529]: pgmap v268: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:43.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:42 smithi092 ceph-mon[32524]: pgmap v268: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:43.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:42 smithi047 ceph-mon[29195]: pgmap v268: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:44.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:44 smithi186 ceph-mon[32529]: pgmap v269: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:45.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:44 smithi092 ceph-mon[32524]: pgmap v269: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:45.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:44 smithi047 ceph-mon[29195]: pgmap v269: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:46.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:46 smithi186 ceph-mon[32529]: pgmap v270: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:47.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:46 smithi092 ceph-mon[32524]: pgmap v270: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:47.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:46 smithi047 ceph-mon[29195]: pgmap v270: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:48.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:48 smithi186 ceph-mon[32529]: pgmap v271: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:49.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:48 smithi092 ceph-mon[32524]: pgmap v271: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:49.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:48 smithi047 ceph-mon[29195]: pgmap v271: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:50.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:50 smithi186 ceph-mon[32529]: pgmap v272: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:51.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:50 smithi092 ceph-mon[32524]: pgmap v272: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:51.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:50 smithi047 ceph-mon[29195]: pgmap v272: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:52.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:52 smithi186 ceph-mon[32529]: pgmap v273: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:53.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:52 smithi092 ceph-mon[32524]: pgmap v273: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:53.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:52 smithi047 ceph-mon[29195]: pgmap v273: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:54.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:54 smithi186 ceph-mon[32529]: pgmap v274: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:55.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:54 smithi092 ceph-mon[32524]: pgmap v274: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:55.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:54 smithi047 ceph-mon[29195]: pgmap v274: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:56.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:56 smithi186 ceph-mon[32529]: pgmap v275: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:57.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:56 smithi092 ceph-mon[32524]: pgmap v275: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:57.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:56 smithi047 ceph-mon[29195]: pgmap v275: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:58.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:08:57 smithi047 ceph-mon[29195]: pgmap v276: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:58.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:08:57 smithi186 ceph-mon[32529]: pgmap v276: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:58.281 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:08:57 smithi092 ceph-mon[32524]: pgmap v276: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:00.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:00 smithi186 ceph-mon[32529]: pgmap v277: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:00.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:00 smithi092 ceph-mon[32524]: pgmap v277: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:00.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:00 smithi047 ceph-mon[29195]: pgmap v277: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:02 smithi186 ceph-mon[32529]: pgmap v278: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:02.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:02 smithi092 ceph-mon[32524]: pgmap v278: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:02.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:02 smithi047 ceph-mon[29195]: pgmap v278: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:04.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:04 smithi092 ceph-mon[32524]: pgmap v279: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:04.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:04 smithi047 ceph-mon[29195]: pgmap v279: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:04.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:04 smithi186 ceph-mon[32529]: pgmap v279: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:06.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:06 smithi092 ceph-mon[32524]: pgmap v280: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:09:06.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:06 smithi047 ceph-mon[29195]: pgmap v280: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:09:06.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:06 smithi186 ceph-mon[32529]: pgmap v280: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:09:08.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:08 smithi092 ceph-mon[32524]: pgmap v281: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:09:08.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:08 smithi047 ceph-mon[29195]: pgmap v281: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:09:08.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:08 smithi186 ceph-mon[32529]: pgmap v281: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:09:09.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:09 smithi092 ceph-mon[32524]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:09:09.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:09 smithi047 ceph-mon[29195]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:09:09.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:09 smithi186 ceph-mon[32529]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:09:10.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:10 smithi092 ceph-mon[32524]: pgmap v283: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:09:10.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:10 smithi047 ceph-mon[29195]: pgmap v283: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:09:10.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:10 smithi186 ceph-mon[32529]: pgmap v283: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:09:12.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:12 smithi092 ceph-mon[32524]: pgmap v284: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:09:12.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:12 smithi092 ceph-mon[32524]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:09:12.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:12 smithi047 ceph-mon[29195]: pgmap v284: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:09:12.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:12 smithi047 ceph-mon[29195]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:09:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:12 smithi186 ceph-mon[32529]: pgmap v284: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:09:12.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:12 smithi186 ceph-mon[32529]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:09:13.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:13 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:09:13.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:13 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:09:13.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:13 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:09:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:14 smithi092 ceph-mon[32524]: pgmap v286: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:09:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:14 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:09:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:14 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:09:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:14 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:09:14.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:14 smithi047 ceph-mon[29195]: pgmap v286: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:09:14.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:14 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:09:14.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:14 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:09:14.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:14 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:09:14.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:14 smithi186 ceph-mon[32529]: pgmap v286: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:09:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:14 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:09:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:14 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:09:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:14 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:09:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:16 smithi092 ceph-mon[32524]: pgmap v287: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:09:16.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:16 smithi047 ceph-mon[29195]: pgmap v287: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:09:16.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:16 smithi186 ceph-mon[32529]: pgmap v287: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:09:18.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:18 smithi092 ceph-mon[32524]: pgmap v288: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-04-25T21:09:18.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:18 smithi047 ceph-mon[29195]: pgmap v288: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-04-25T21:09:18.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:18 smithi186 ceph-mon[32529]: pgmap v288: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-04-25T21:09:20.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:20 smithi092 ceph-mon[32524]: pgmap v289: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:09:20.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:20 smithi047 ceph-mon[29195]: pgmap v289: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:09:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:20 smithi186 ceph-mon[32529]: pgmap v289: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:09:22.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:22 smithi092 ceph-mon[32524]: pgmap v290: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:09:22.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:22 smithi047 ceph-mon[29195]: pgmap v290: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:09:22.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:22 smithi186 ceph-mon[32529]: pgmap v290: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:09:24.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:24 smithi092 ceph-mon[32524]: pgmap v291: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:09:24.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:24 smithi047 ceph-mon[29195]: pgmap v291: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:09:24.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:24 smithi186 ceph-mon[32529]: pgmap v291: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:09:26.522 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:26 smithi186 ceph-mon[32529]: pgmap v292: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:09:26.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:26 smithi092 ceph-mon[32524]: pgmap v292: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:09:26.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:26 smithi047 ceph-mon[29195]: pgmap v292: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:09:28.531 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:28 smithi092 ceph-mon[32524]: pgmap v293: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:28 smithi047 ceph-mon[29195]: pgmap v293: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:28.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:28 smithi186 ceph-mon[32529]: pgmap v293: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:30.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:30 smithi092 ceph-mon[32524]: pgmap v294: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:30.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:30 smithi047 ceph-mon[29195]: pgmap v294: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:30.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:30 smithi186 ceph-mon[32529]: pgmap v294: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:32.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:32 smithi092 ceph-mon[32524]: pgmap v295: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:32.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:32 smithi047 ceph-mon[29195]: pgmap v295: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:32.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:32 smithi186 ceph-mon[32529]: pgmap v295: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:34.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:34 smithi092 ceph-mon[32524]: pgmap v296: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:34.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:34 smithi047 ceph-mon[29195]: pgmap v296: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:34.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:34 smithi186 ceph-mon[32529]: pgmap v296: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:36.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:36 smithi092 ceph-mon[32524]: pgmap v297: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:36.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:36 smithi047 ceph-mon[29195]: pgmap v297: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:36.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:36 smithi186 ceph-mon[32529]: pgmap v297: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:38.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:38 smithi047 ceph-mon[29195]: pgmap v298: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:38.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:38 smithi186 ceph-mon[32529]: pgmap v298: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:38.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:38 smithi092 ceph-mon[32524]: pgmap v298: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:40.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:40 smithi047 ceph-mon[29195]: pgmap v299: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:40.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:40 smithi186 ceph-mon[32529]: pgmap v299: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:40.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:40 smithi092 ceph-mon[32524]: pgmap v299: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:42.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:42 smithi047 ceph-mon[29195]: pgmap v300: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:42.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:42 smithi186 ceph-mon[32529]: pgmap v300: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:42.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:42 smithi092 ceph-mon[32524]: pgmap v300: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:44.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:44 smithi047 ceph-mon[29195]: pgmap v301: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:44.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:44 smithi186 ceph-mon[32529]: pgmap v301: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:44.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:44 smithi092 ceph-mon[32524]: pgmap v301: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:46.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:46 smithi047 ceph-mon[29195]: pgmap v302: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:46.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:46 smithi186 ceph-mon[32529]: pgmap v302: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:46.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:46 smithi092 ceph-mon[32524]: pgmap v302: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:48.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:48 smithi047 ceph-mon[29195]: pgmap v303: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:48.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:48 smithi186 ceph-mon[32529]: pgmap v303: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:48.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:48 smithi092 ceph-mon[32524]: pgmap v303: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:50.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:50 smithi047 ceph-mon[29195]: pgmap v304: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:50 smithi186 ceph-mon[32529]: pgmap v304: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:50.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:50 smithi092 ceph-mon[32524]: pgmap v304: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:52.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:52 smithi047 ceph-mon[29195]: pgmap v305: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:52 smithi186 ceph-mon[32529]: pgmap v305: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:52.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:52 smithi092 ceph-mon[32524]: pgmap v305: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:54.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:54 smithi047 ceph-mon[29195]: pgmap v306: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:54 smithi186 ceph-mon[32529]: pgmap v306: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:54 smithi092 ceph-mon[32524]: pgmap v306: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:56.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:56 smithi047 ceph-mon[29195]: pgmap v307: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:56 smithi186 ceph-mon[32529]: pgmap v307: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:56 smithi092 ceph-mon[32524]: pgmap v307: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:09:58.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:09:58 smithi047 ceph-mon[29195]: pgmap v308: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:09:58 smithi186 ceph-mon[32529]: pgmap v308: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:09:58.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:09:58 smithi092 ceph-mon[32524]: pgmap v308: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:00.583 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:00 smithi092 ceph-mon[32524]: pgmap v309: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:00.583 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:00 smithi092 ceph-mon[32524]: overall HEALTH_OK 2024-04-25T21:10:00.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:00 smithi047 ceph-mon[29195]: pgmap v309: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:00.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:00 smithi047 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T21:10:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:00 smithi186 ceph-mon[32529]: pgmap v309: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:00 smithi186 ceph-mon[32529]: overall HEALTH_OK 2024-04-25T21:10:01.776 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T21:10:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 -- 172.21.15.47:0/4172181945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 msgr2=0x7fb434074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/4172181945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 0x7fb434074760 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009900 tx=0x7fb42c02a480 comp rx=0 tx=0).stop 2024-04-25T21:10:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 -- 172.21.15.47:0/4172181945 shutdown_connections 2024-04-25T21:10:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/4172181945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb43410e770 0x7fb434110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/4172181945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb434074ca0 0x7fb43410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/4172181945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 0x7fb434074760 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 -- 172.21.15.47:0/4172181945 >> 172.21.15.47:0/4172181945 conn(0x7fb4340fda00 msgr2=0x7fb4340ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 -- 172.21.15.47:0/4172181945 shutdown_connections 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.896+0000 7fb43b6c2640 1 -- 172.21.15.47:0/4172181945 wait complete. 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43b6c2640 1 Processor -- start 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43b6c2640 1 -- start start 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43b6c2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 0x7fb4341a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43b6c2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb434074ca0 0x7fb4341a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43b6c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43410e770 0x7fb43419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43b6c2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341143a0 con 0x7fb434074380 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43b6c2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb434114220 con 0x7fb43410e770 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43b6c2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb434114520 con 0x7fb434074ca0 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43945e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 0x7fb4341a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43945e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 0x7fb4341a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38248/0 (socket says 172.21.15.47:38248) 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43945e640 1 -- 172.21.15.47:0/541106132 learned_addr learned my addr 172.21.15.47:0/541106132 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb438c5d640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb434074ca0 0x7fb4341a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb439c5f640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43410e770 0x7fb43419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43945e640 1 -- 172.21.15.47:0/541106132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb434074ca0 msgr2=0x7fb4341a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43945e640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb434074ca0 0x7fb4341a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43945e640 1 -- 172.21.15.47:0/541106132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43410e770 msgr2=0x7fb43419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43945e640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43410e770 0x7fb43419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.897+0000 7fb43945e640 1 -- 172.21.15.47:0/541106132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43419af60 con 0x7fb434074380 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.898+0000 7fb439c5f640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43410e770 0x7fb43419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.898+0000 7fb438c5d640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb434074ca0 0x7fb4341a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.898+0000 7fb43945e640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 0x7fb4341a05e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb42c005940 tx=0x7fb42c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.898+0000 7fb42a7fc640 1 -- 172.21.15.47:0/541106132 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c03f030 con 0x7fb434074380 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.898+0000 7fb42a7fc640 1 -- 172.21.15.47:0/541106132 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb42c003cb0 con 0x7fb434074380 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.898+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43419b1f0 con 0x7fb434074380 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.898+0000 7fb42a7fc640 1 -- 172.21.15.47:0/541106132 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c035070 con 0x7fb434074380 2024-04-25T21:10:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.898+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4341a7380 con 0x7fb434074380 2024-04-25T21:10:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.899+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb434075a30 con 0x7fb434074380 2024-04-25T21:10:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.901+0000 7fb42a7fc640 1 -- 172.21.15.47:0/541106132 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb42c042030 con 0x7fb434074380 2024-04-25T21:10:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.902+0000 7fb42a7fc640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4200a3dc0 0x7fb4200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.902+0000 7fb438c5d640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4200a3dc0 0x7fb4200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.902+0000 7fb42a7fc640 1 -- 172.21.15.47:0/541106132 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7fb42c0f13d0 con 0x7fb434074380 2024-04-25T21:10:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.902+0000 7fb438c5d640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4200a3dc0 0x7fb4200a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb43419bef0 tx=0x7fb424005e50 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:01.905+0000 7fb42a7fc640 1 -- 172.21.15.47:0/541106132 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb42c0bd9f0 con 0x7fb434074380 2024-04-25T21:10:02.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.117+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb434062390 con 0x7fb4200a3dc0 2024-04-25T21:10:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.125+0000 7fb42a7fc640 1 -- 172.21.15.47:0/541106132 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fb434062390 con 0x7fb4200a3dc0 2024-04-25T21:10:02.125 INFO:teuthology.orchestra.run.smithi047.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T21:00:00", "first": "2024-04-25T21:01:00", "last": "2024-04-25T21:10:00", "last_pruned": "2024-04-25T21:10:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T21:10:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4200a3dc0 msgr2=0x7fb4200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4200a3dc0 0x7fb4200a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb43419bef0 tx=0x7fb424005e50 comp rx=0 tx=0).stop 2024-04-25T21:10:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 msgr2=0x7fb4341a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 0x7fb4341a05e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb42c005940 tx=0x7fb42c0042f0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 shutdown_connections 2024-04-25T21:10:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb4200a3dc0 0x7fb4200a6280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43410e770 0x7fb43419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb434074ca0 0x7fb4341a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 --2- 172.21.15.47:0/541106132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb434074380 0x7fb4341a05e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 >> 172.21.15.47:0/541106132 conn(0x7fb4340fda00 msgr2=0x7fb4340fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 shutdown_connections 2024-04-25T21:10:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.127+0000 7fb43b6c2640 1 -- 172.21.15.47:0/541106132 wait complete. 2024-04-25T21:10:02.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T21:10:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 -- 172.21.15.47:0/3539364201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 msgr2=0x7f540c107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 --2- 172.21.15.47:0/3539364201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 0x7f540c107390 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5408009900 tx=0x7f540802a400 comp rx=0 tx=0).stop 2024-04-25T21:10:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 -- 172.21.15.47:0/3539364201 shutdown_connections 2024-04-25T21:10:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 --2- 172.21.15.47:0/3539364201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c101b60 0x7f540c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 --2- 172.21.15.47:0/3539364201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c1011a0 0x7f540c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 --2- 172.21.15.47:0/3539364201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 0x7f540c107390 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 -- 172.21.15.47:0/3539364201 >> 172.21.15.47:0/3539364201 conn(0x7f540c076c50 msgr2=0x7f540c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 -- 172.21.15.47:0/3539364201 shutdown_connections 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.241+0000 7f541435d640 1 -- 172.21.15.47:0/3539364201 wait complete. 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f541435d640 1 Processor -- start 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f541435d640 1 -- start start 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f541435d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c1011a0 0x7f540c102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f541435d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c101b60 0x7f540c103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f541435d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 0x7f540c103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f541435d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f540c111090 con 0x7f540c106fb0 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f541435d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f540c110f10 con 0x7f540c1011a0 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f541435d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f540c111210 con 0x7f540c101b60 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f54128fa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 0x7f540c103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f54128fa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 0x7f540c103590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38270/0 (socket says 172.21.15.47:38270) 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f54128fa640 1 -- 172.21.15.47:0/2518856791 learned_addr learned my addr 172.21.15.47:0/2518856791 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f54118f8640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c101b60 0x7f540c103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.242+0000 7f54120f9640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c1011a0 0x7f540c102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54128fa640 1 -- 172.21.15.47:0/2518856791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c101b60 msgr2=0x7f540c103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54128fa640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c101b60 0x7f540c103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54128fa640 1 -- 172.21.15.47:0/2518856791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c1011a0 msgr2=0x7f540c102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54128fa640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c1011a0 0x7f540c102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54128fa640 1 -- 172.21.15.47:0/2518856791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c1a6b60 con 0x7f540c106fb0 2024-04-25T21:10:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54128fa640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 0x7f540c103590 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f540400eaa0 tx=0x7f540400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54037fe640 1 -- 172.21.15.47:0/2518856791 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400cdb0 con 0x7f540c106fb0 2024-04-25T21:10:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54037fe640 1 -- 172.21.15.47:0/2518856791 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5404005020 con 0x7f540c106fb0 2024-04-25T21:10:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c1a6e50 con 0x7f540c106fb0 2024-04-25T21:10:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f54037fe640 1 -- 172.21.15.47:0/2518856791 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54040116d0 con 0x7f540c106fb0 2024-04-25T21:10:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.243+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a7320 con 0x7f540c106fb0 2024-04-25T21:10:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.244+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c102550 con 0x7f540c106fb0 2024-04-25T21:10:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.245+0000 7f54037fe640 1 -- 172.21.15.47:0/2518856791 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f54040051d0 con 0x7f540c106fb0 2024-04-25T21:10:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.246+0000 7f54037fe640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53f80a3dc0 0x7f53f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.246+0000 7f54037fe640 1 -- 172.21.15.47:0/2518856791 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7f54040d09c0 con 0x7f540c106fb0 2024-04-25T21:10:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.246+0000 7f54120f9640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53f80a3dc0 0x7f53f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.246+0000 7f54120f9640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53f80a3dc0 0x7f53f80a6280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5408031440 tx=0x7f54080095f0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.250+0000 7f54037fe640 1 -- 172.21.15.47:0/2518856791 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54040d2e20 con 0x7f540c106fb0 2024-04-25T21:10:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.464+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 --> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f540c105c60 con 0x7f53f80a3dc0 2024-04-25T21:10:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.471+0000 7f54037fe640 1 -- 172.21.15.47:0/2518856791 <== mgr.24811 v2:172.21.15.47:6800/957595556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f540c105c60 con 0x7f53f80a3dc0 2024-04-25T21:10:02.471 INFO:teuthology.orchestra.run.smithi047.stdout:/ 1m 6m3h 2024-04-25T21:10:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53f80a3dc0 msgr2=0x7f53f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53f80a3dc0 0x7f53f80a6280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5408031440 tx=0x7f54080095f0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 msgr2=0x7f540c103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 0x7f540c103590 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f540400eaa0 tx=0x7f540400ef60 comp rx=0 tx=0).stop 2024-04-25T21:10:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 shutdown_connections 2024-04-25T21:10:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c106fb0 0x7f540c103590 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c101b60 0x7f540c103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f53f80a3dc0 0x7f53f80a6280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 --2- 172.21.15.47:0/2518856791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c1011a0 0x7f540c102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 >> 172.21.15.47:0/2518856791 conn(0x7f540c076c50 msgr2=0x7f540c06a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 shutdown_connections 2024-04-25T21:10:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.474+0000 7f541435d640 1 -- 172.21.15.47:0/2518856791 wait complete. 2024-04-25T21:10:02.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:02 smithi047 ceph-mon[29195]: pgmap v310: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:02.485 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T21:10:02.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T21:10:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 -- 172.21.15.47:0/3720996823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 msgr2=0x7f1638102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 --2- 172.21.15.47:0/3720996823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 0x7f1638102620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1634009ca0 tx=0x7f163402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 -- 172.21.15.47:0/3720996823 shutdown_connections 2024-04-25T21:10:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 --2- 172.21.15.47:0/3720996823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1638107280 0x7f163810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 --2- 172.21.15.47:0/3720996823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1638102bf0 0x7f1638106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 --2- 172.21.15.47:0/3720996823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 0x7f1638102620 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 -- 172.21.15.47:0/3720996823 >> 172.21.15.47:0/3720996823 conn(0x7f16380790e0 msgr2=0x7f16380794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 -- 172.21.15.47:0/3720996823 shutdown_connections 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.600+0000 7f16404cb640 1 -- 172.21.15.47:0/3720996823 wait complete. 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f16404cb640 1 Processor -- start 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f16404cb640 1 -- start start 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f16404cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 0x7f16381986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f16404cb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1638107280 0x7f1638198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f16404cb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f163819cfa0 0x7f163819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f16404cb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f16381121e0 con 0x7f1638102240 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f16404cb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1638112060 con 0x7f1638107280 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f16404cb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1638112360 con 0x7f163819cfa0 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 0x7f16381986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 0x7f16381986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38286/0 (socket says 172.21.15.47:38286) 2024-04-25T21:10:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 -- 172.21.15.47:0/86699867 learned_addr learned my addr 172.21.15.47:0/86699867 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163ea68640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f163819cfa0 0x7f163819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163da66640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1638107280 0x7f1638198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 -- 172.21.15.47:0/86699867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f163819cfa0 msgr2=0x7f163819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f163819cfa0 0x7f163819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 -- 172.21.15.47:0/86699867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1638107280 msgr2=0x7f1638198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1638107280 0x7f1638198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 -- 172.21.15.47:0/86699867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163819da90 con 0x7f1638102240 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.601+0000 7f163e267640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 0x7f16381986e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1634006010 tx=0x7f16340319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.602+0000 7f162f7fe640 1 -- 172.21.15.47:0/86699867 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634003bf0 con 0x7f1638102240 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.602+0000 7f162f7fe640 1 -- 172.21.15.47:0/86699867 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1634003d90 con 0x7f1638102240 2024-04-25T21:10:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.602+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163819dd20 con 0x7f1638102240 2024-04-25T21:10:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.602+0000 7f162f7fe640 1 -- 172.21.15.47:0/86699867 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163402fc20 con 0x7f1638102240 2024-04-25T21:10:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.602+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16381a5570 con 0x7f1638102240 2024-04-25T21:10:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.603+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16380691c0 con 0x7f1638102240 2024-04-25T21:10:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.604+0000 7f162f7fe640 1 -- 172.21.15.47:0/86699867 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f16340046a0 con 0x7f1638102240 2024-04-25T21:10:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.605+0000 7f162f7fe640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f16240a3da0 0x7f16240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.605+0000 7f162f7fe640 1 -- 172.21.15.47:0/86699867 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7f16340f1d90 con 0x7f1638102240 2024-04-25T21:10:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.605+0000 7f163da66640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f16240a3da0 0x7f16240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.605+0000 7f163da66640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f16240a3da0 0x7f16240a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1628005fd0 tx=0x7f1628005850 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.608+0000 7f162f7fe640 1 -- 172.21.15.47:0/86699867 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16340be520 con 0x7f1638102240 2024-04-25T21:10:02.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:02 smithi186 ceph-mon[32529]: pgmap v310: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:02.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:02 smithi092 ceph-mon[32524]: pgmap v310: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:02.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.796+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f163810a480 con 0x7f1638102240 2024-04-25T21:10:02.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.797+0000 7f162f7fe640 1 -- 172.21.15.47:0/86699867 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23064 (secure 0 0 0) 0x7f16340bde20 con 0x7f1638102240 2024-04-25T21:10:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:10:02.798 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":98,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T21:09:11.831136+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1752629947":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/831854467":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/2517449296":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6801/253800519":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6800/253800519":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/3318702222":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1},{"begin":13,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:10:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f16240a3da0 msgr2=0x7f16240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f16240a3da0 0x7f16240a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1628005fd0 tx=0x7f1628005850 comp rx=0 tx=0).stop 2024-04-25T21:10:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 msgr2=0x7f16381986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 0x7f16381986e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1634006010 tx=0x7f16340319f0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 shutdown_connections 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f163819cfa0 0x7f163819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f16240a3da0 0x7f16240a6260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1638107280 0x7f1638198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 --2- 172.21.15.47:0/86699867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1638102240 0x7f16381986e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.799+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 >> 172.21.15.47:0/86699867 conn(0x7f16380790e0 msgr2=0x7f1638100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.800+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 shutdown_connections 2024-04-25T21:10:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.800+0000 7f16404cb640 1 -- 172.21.15.47:0/86699867 wait complete. 2024-04-25T21:10:02.811 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T20:52:09.901826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T20:56:46.327630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T20:56:49.911562+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T21:10:02.812 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T21:10:02.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 -- 172.21.15.47:0/1300113700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a810e7f0 msgr2=0x7f03a8110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/1300113700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a810e7f0 0x7f03a8110be0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009ca0 tx=0x7f03a002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 -- 172.21.15.47:0/1300113700 shutdown_connections 2024-04-25T21:10:02.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/1300113700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a810e7f0 0x7f03a8110be0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/1300113700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03a81014f0 0x7f03a810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/1300113700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a8100b40 0x7f03a8100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 -- 172.21.15.47:0/1300113700 >> 172.21.15.47:0/1300113700 conn(0x7f03a8076150 msgr2=0x7f03a8076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 -- 172.21.15.47:0/1300113700 shutdown_connections 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.917+0000 7f03ae6fd640 1 -- 172.21.15.47:0/1300113700 wait complete. 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.918+0000 7f03ae6fd640 1 Processor -- start 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.918+0000 7f03ae6fd640 1 -- start start 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.918+0000 7f03ae6fd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a8100b40 0x7f03a81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.918+0000 7f03ae6fd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a81014f0 0x7f03a81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.918+0000 7f03ae6fd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03a810e7f0 0x7f03a819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.918+0000 7f03ae6fd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f03a81143a0 con 0x7f03a81014f0 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.918+0000 7f03ae6fd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8114220 con 0x7f03a810e7f0 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.918+0000 7f03ae6fd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8114520 con 0x7f03a8100b40 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a81014f0 0x7f03a81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a81014f0 0x7f03a81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38304/0 (socket says 172.21.15.47:38304) 2024-04-25T21:10:02.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 -- 172.21.15.47:0/3484557539 learned_addr learned my addr 172.21.15.47:0/3484557539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03acc9a640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03a810e7f0 0x7f03a819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a7fff640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a8100b40 0x7f03a81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 -- 172.21.15.47:0/3484557539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a8100b40 msgr2=0x7f03a81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a8100b40 0x7f03a81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 -- 172.21.15.47:0/3484557539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03a810e7f0 msgr2=0x7f03a819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03a810e7f0 0x7f03a819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 -- 172.21.15.47:0/3484557539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a819b020 con 0x7f03a81014f0 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a7fff640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a8100b40 0x7f03a81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03acc9a640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03a810e7f0 0x7f03a819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a77fe640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a81014f0 0x7f03a81a0be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e9f0 tx=0x7f039c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a57fa640 1 -- 172.21.15.47:0/3484557539 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c00cd80 con 0x7f03a81014f0 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a819b310 con 0x7f03a81014f0 2024-04-25T21:10:02.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a57fa640 1 -- 172.21.15.47:0/3484557539 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039c0108e0 con 0x7f03a81014f0 2024-04-25T21:10:02.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03a57fa640 1 -- 172.21.15.47:0/3484557539 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c018980 con 0x7f03a81014f0 2024-04-25T21:10:02.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.919+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a81a7620 con 0x7f03a81014f0 2024-04-25T21:10:02.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.920+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a8069eb0 con 0x7f03a81014f0 2024-04-25T21:10:02.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.921+0000 7f03a57fa640 1 -- 172.21.15.47:0/3484557539 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f039c010430 con 0x7f03a81014f0 2024-04-25T21:10:02.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.922+0000 7f03a57fa640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f03900a3d20 0x7f03900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.922+0000 7f03a7fff640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f03900a3d20 0x7f03900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.922+0000 7f03a57fa640 1 -- 172.21.15.47:0/3484557539 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7f039c0d0c40 con 0x7f03a81014f0 2024-04-25T21:10:02.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.922+0000 7f03a7fff640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f03900a3d20 0x7f03900a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0398009b40 tx=0x7f0398005e50 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:02.926+0000 7f03a57fa640 1 -- 172.21.15.47:0/3484557539 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f039c09d260 con 0x7f03a81014f0 2024-04-25T21:10:03.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.111+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03a81098f0 con 0x7f03a81014f0 2024-04-25T21:10:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.112+0000 7f03a57fa640 1 -- 172.21.15.47:0/3484557539 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f039c09cb60 con 0x7f03a81014f0 2024-04-25T21:10:03.112 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-25T21:10:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f03900a3d20 msgr2=0x7f03900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f03900a3d20 0x7f03900a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0398009b40 tx=0x7f0398005e50 comp rx=0 tx=0).stop 2024-04-25T21:10:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a81014f0 msgr2=0x7f03a81a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a81014f0 0x7f03a81a0be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e9f0 tx=0x7f039c00eeb0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 shutdown_connections 2024-04-25T21:10:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f03900a3d20 0x7f03900a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03a810e7f0 0x7f03a819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a81014f0 0x7f03a81a0be0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 --2- 172.21.15.47:0/3484557539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a8100b40 0x7f03a81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 >> 172.21.15.47:0/3484557539 conn(0x7f03a8076150 msgr2=0x7f03a810ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 shutdown_connections 2024-04-25T21:10:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.114+0000 7f03ae6fd640 1 -- 172.21.15.47:0/3484557539 wait complete. 2024-04-25T21:10:03.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T21:10:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/4063574604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 msgr2=0x7fdb54101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/4063574604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 0x7fdb54101460 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009900 tx=0x7fdb5002a410 comp rx=0 tx=0).stop 2024-04-25T21:10:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/4063574604 shutdown_connections 2024-04-25T21:10:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/4063574604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb54106150 0x7fdb5410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/4063574604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb54101a30 0x7fdb54105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/4063574604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 0x7fdb54101460 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/4063574604 >> 172.21.15.47:0/4063574604 conn(0x7fdb54078000 msgr2=0x7fdb54078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/4063574604 shutdown_connections 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/4063574604 wait complete. 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 Processor -- start 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.231+0000 7fdb5c6cb640 1 -- start start 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5c6cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 0x7fdb541986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5c6cb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb54106150 0x7fdb54198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5c6cb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5419cfb0 0x7fdb5419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5c6cb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541121d0 con 0x7fdb54101080 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5c6cb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54112050 con 0x7fdb54106150 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5c6cb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54112350 con 0x7fdb5419cfb0 2024-04-25T21:10:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 0x7fdb541986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 0x7fdb541986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38316/0 (socket says 172.21.15.47:38316) 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 -- 172.21.15.47:0/1304201217 learned_addr learned my addr 172.21.15.47:0/1304201217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb59c66640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb54106150 0x7fdb54198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5ac68640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5419cfb0 0x7fdb5419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 -- 172.21.15.47:0/1304201217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5419cfb0 msgr2=0x7fdb5419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5419cfb0 0x7fdb5419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 -- 172.21.15.47:0/1304201217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb54106150 msgr2=0x7fdb54198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb54106150 0x7fdb54198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 -- 172.21.15.47:0/1304201217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5419daa0 con 0x7fdb54101080 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb59c66640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb54106150 0x7fdb54198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5a467640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 0x7fdb541986f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdb50005940 tx=0x7fdb50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb4b7fe640 1 -- 172.21.15.47:0/1304201217 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5003f030 con 0x7fdb54101080 2024-04-25T21:10:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb4b7fe640 1 -- 172.21.15.47:0/1304201217 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb50003aa0 con 0x7fdb54101080 2024-04-25T21:10:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5419dd30 con 0x7fdb54101080 2024-04-25T21:10:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb4b7fe640 1 -- 172.21.15.47:0/1304201217 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50035070 con 0x7fdb54101080 2024-04-25T21:10:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.232+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb541a54f0 con 0x7fdb54101080 2024-04-25T21:10:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.234+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb54068d70 con 0x7fdb54101080 2024-04-25T21:10:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.235+0000 7fdb4b7fe640 1 -- 172.21.15.47:0/1304201217 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdb50042030 con 0x7fdb54101080 2024-04-25T21:10:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.235+0000 7fdb4b7fe640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdb400a3d20 0x7fdb400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.235+0000 7fdb59c66640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdb400a3d20 0x7fdb400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.235+0000 7fdb4b7fe640 1 -- 172.21.15.47:0/1304201217 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7fdb500f1270 con 0x7fdb54101080 2024-04-25T21:10:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.236+0000 7fdb59c66640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdb400a3d20 0x7fdb400a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb541009c0 tx=0x7fdb4400b5f0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.239+0000 7fdb4b7fe640 1 -- 172.21.15.47:0/1304201217 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb500bd8f0 con 0x7fdb54101080 2024-04-25T21:10:03.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.429+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb54101a30 con 0x7fdb54101080 2024-04-25T21:10:03.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.429+0000 7fdb4b7fe640 1 -- 172.21.15.47:0/1304201217 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fdb500bd1f0 con 0x7fdb54101080 2024-04-25T21:10:03.429 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-25T21:10:03.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdb400a3d20 msgr2=0x7fdb400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdb400a3d20 0x7fdb400a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb541009c0 tx=0x7fdb4400b5f0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 msgr2=0x7fdb541986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 0x7fdb541986f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdb50005940 tx=0x7fdb50004270 comp rx=0 tx=0).stop 2024-04-25T21:10:03.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 shutdown_connections 2024-04-25T21:10:03.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5419cfb0 0x7fdb5419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdb400a3d20 0x7fdb400a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb54106150 0x7fdb54198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 --2- 172.21.15.47:0/1304201217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb54101080 0x7fdb541986f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 >> 172.21.15.47:0/1304201217 conn(0x7fdb54078000 msgr2=0x7fdb540ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 shutdown_connections 2024-04-25T21:10:03.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.431+0000 7fdb5c6cb640 1 -- 172.21.15.47:0/1304201217 wait complete. 2024-04-25T21:10:03.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:03 smithi047 ceph-mon[29195]: from='client.15909 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:10:03.441 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:03 smithi047 ceph-mon[29195]: from='client.15915 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:10:03.441 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:03 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/86699867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:10:03.441 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:03 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3484557539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:10:03.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.546+0000 7fa6ae508640 1 -- 172.21.15.47:0/276086091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 msgr2=0x7fa6a810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.546+0000 7fa6ae508640 1 --2- 172.21.15.47:0/276086091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 0x7fa6a810dd30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009ca0 tx=0x7fa6a002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 -- 172.21.15.47:0/276086091 shutdown_connections 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 --2- 172.21.15.47:0/276086091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 0x7fa6a810dd30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 --2- 172.21.15.47:0/276086091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a8101a40 0x7fa6a8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 --2- 172.21.15.47:0/276086091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8101090 0x7fa6a8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 -- 172.21.15.47:0/276086091 >> 172.21.15.47:0/276086091 conn(0x7fa6a8077f50 msgr2=0x7fa6a8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 -- 172.21.15.47:0/276086091 shutdown_connections 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 -- 172.21.15.47:0/276086091 wait complete. 2024-04-25T21:10:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 Processor -- start 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.547+0000 7fa6ae508640 1 -- start start 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6ae508640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8101090 0x7fa6a81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6ae508640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 0x7fa6a8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6ae508640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a819a070 0x7fa6a81a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6ae508640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a8111120 con 0x7fa6a8106160 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6ae508640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a8110fa0 con 0x7fa6a8101090 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6ae508640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81112a0 con 0x7fa6a819a070 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 0x7fa6a8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 0x7fa6a8199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38344/0 (socket says 172.21.15.47:38344) 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 -- 172.21.15.47:0/2121750230 learned_addr learned my addr 172.21.15.47:0/2121750230 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a7fff640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8101090 0x7fa6a81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6acaa5640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a819a070 0x7fa6a81a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 -- 172.21.15.47:0/2121750230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a819a070 msgr2=0x7fa6a81a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a819a070 0x7fa6a81a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 -- 172.21.15.47:0/2121750230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8101090 msgr2=0x7fa6a81995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8101090 0x7fa6a81995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 -- 172.21.15.47:0/2121750230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a81a4e20 con 0x7fa6a8106160 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a77fe640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 0x7fa6a8199ae0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa69000b450 tx=0x7fa69000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a57fa640 1 -- 172.21.15.47:0/2121750230 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690015020 con 0x7fa6a8106160 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a57fa640 1 -- 172.21.15.47:0/2121750230 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa690011d40 con 0x7fa6a8106160 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a81a5110 con 0x7fa6a8106160 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6a57fa640 1 -- 172.21.15.47:0/2121750230 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69000f340 con 0x7fa6a8106160 2024-04-25T21:10:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.548+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a81a56b0 con 0x7fa6a8106160 2024-04-25T21:10:03.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.549+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a8068d70 con 0x7fa6a8106160 2024-04-25T21:10:03.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.551+0000 7fa6a57fa640 1 -- 172.21.15.47:0/2121750230 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa690018020 con 0x7fa6a8106160 2024-04-25T21:10:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.551+0000 7fa6a57fa640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa6940a3da0 0x7fa6940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.551+0000 7fa6a57fa640 1 -- 172.21.15.47:0/2121750230 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7fa6900d0140 con 0x7fa6a8106160 2024-04-25T21:10:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.551+0000 7fa6a7fff640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa6940a3da0 0x7fa6940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.552+0000 7fa6a7fff640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa6940a3da0 0x7fa6940a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa69800bbe0 tx=0x7fa69800b390 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:03.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.556+0000 7fa6a57fa640 1 -- 172.21.15.47:0/2121750230 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa69001f020 con 0x7fa6a8106160 2024-04-25T21:10:03.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:03 smithi186 ceph-mon[32529]: from='client.15909 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:10:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:03 smithi186 ceph-mon[32529]: from='client.15915 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:10:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:03 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/86699867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:10:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:03 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3484557539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:10:03.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.740+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6a8109390 con 0x7fa6a8106160 2024-04-25T21:10:03.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.741+0000 7fa6a57fa640 1 -- 172.21.15.47:0/2121750230 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7fa69000d040 con 0x7fa6a8106160 2024-04-25T21:10:03.741 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa6940a3da0 msgr2=0x7fa6940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa6940a3da0 0x7fa6940a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa69800bbe0 tx=0x7fa69800b390 comp rx=0 tx=0).stop 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 msgr2=0x7fa6a8199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 0x7fa6a8199ae0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa69000b450 tx=0x7fa69000b910 comp rx=0 tx=0).stop 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 shutdown_connections 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a819a070 0x7fa6a81a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6a8106160 0x7fa6a8199ae0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa6940a3da0 0x7fa6940a6260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 --2- 172.21.15.47:0/2121750230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8101090 0x7fa6a81995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 >> 172.21.15.47:0/2121750230 conn(0x7fa6a8077f50 msgr2=0x7fa6a80ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:03.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 shutdown_connections 2024-04-25T21:10:03.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.743+0000 7fa6ae508640 1 -- 172.21.15.47:0/2121750230 wait complete. 2024-04-25T21:10:03.754 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T21:10:03.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:03 smithi092 ceph-mon[32524]: from='client.15909 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:10:03.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:03 smithi092 ceph-mon[32524]: from='client.15915 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:10:03.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:03 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/86699867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:10:03.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:03 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3484557539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:10:03.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 -- 172.21.15.47:0/245739015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c10c570 msgr2=0x7fbe5c10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 --2- 172.21.15.47:0/245739015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c10c570 0x7fbe5c10e9e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009ca0 tx=0x7fbe5402a950 comp rx=0 tx=0).stop 2024-04-25T21:10:03.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 -- 172.21.15.47:0/245739015 shutdown_connections 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 --2- 172.21.15.47:0/245739015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c10c570 0x7fbe5c10e9e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 --2- 172.21.15.47:0/245739015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5c103b50 0x7fbe5c10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 --2- 172.21.15.47:0/245739015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5c103230 0x7fbe5c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 -- 172.21.15.47:0/245739015 >> 172.21.15.47:0/245739015 conn(0x7fbe5c077fb0 msgr2=0x7fbe5c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 -- 172.21.15.47:0/245739015 shutdown_connections 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.856+0000 7fbe64108640 1 -- 172.21.15.47:0/245739015 wait complete. 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe64108640 1 Processor -- start 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe64108640 1 -- start start 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe64108640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c103230 0x7fbe5c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe64108640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5c103b50 0x7fbe5c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe64108640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5c10c570 0x7fbe5c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe64108640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1121b0 con 0x7fbe5c103230 2024-04-25T21:10:03.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe64108640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c112030 con 0x7fbe5c10c570 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe64108640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c112330 con 0x7fbe5c103b50 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe61ea4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c103230 0x7fbe5c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe61ea4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c103230 0x7fbe5c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39698/0 (socket says 172.21.15.47:39698) 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe61ea4640 1 -- 172.21.15.47:0/2204371764 learned_addr learned my addr 172.21.15.47:0/2204371764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe616a3640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5c103b50 0x7fbe5c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe626a5640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5c10c570 0x7fbe5c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe61ea4640 1 -- 172.21.15.47:0/2204371764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5c103b50 msgr2=0x7fbe5c198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe61ea4640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5c103b50 0x7fbe5c198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe61ea4640 1 -- 172.21.15.47:0/2204371764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5c10c570 msgr2=0x7fbe5c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.857+0000 7fbe61ea4640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5c10c570 0x7fbe5c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.858+0000 7fbe61ea4640 1 -- 172.21.15.47:0/2204371764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c19d640 con 0x7fbe5c103230 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.858+0000 7fbe626a5640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5c10c570 0x7fbe5c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.858+0000 7fbe61ea4640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c103230 0x7fbe5c198650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c920 tx=0x7fbe5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.858+0000 7fbe52ffd640 1 -- 172.21.15.47:0/2204371764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58007c70 con 0x7fbe5c103230 2024-04-25T21:10:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.858+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c19d930 con 0x7fbe5c103230 2024-04-25T21:10:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.858+0000 7fbe52ffd640 1 -- 172.21.15.47:0/2204371764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe58007e10 con 0x7fbe5c103230 2024-04-25T21:10:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.858+0000 7fbe52ffd640 1 -- 172.21.15.47:0/2204371764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58018820 con 0x7fbe5c103230 2024-04-25T21:10:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.858+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c1a51c0 con 0x7fbe5c103230 2024-04-25T21:10:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.859+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c0691c0 con 0x7fbe5c103230 2024-04-25T21:10:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.860+0000 7fbe52ffd640 1 -- 172.21.15.47:0/2204371764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbe5800f450 con 0x7fbe5c103230 2024-04-25T21:10:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.860+0000 7fbe52ffd640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbe480a3d20 0x7fbe480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.861+0000 7fbe616a3640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbe480a3d20 0x7fbe480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.861+0000 7fbe52ffd640 1 -- 172.21.15.47:0/2204371764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7fbe580d09f0 con 0x7fbe5c103230 2024-04-25T21:10:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.861+0000 7fbe616a3640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbe480a3d20 0x7fbe480a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c10e270 tx=0x7fbe4c005f90 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:03.865+0000 7fbe52ffd640 1 -- 172.21.15.47:0/2204371764 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe58014030 con 0x7fbe5c103230 2024-04-25T21:10:04.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.100+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fbe5c1a5420 con 0x7fbe5c103230 2024-04-25T21:10:04.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.108+0000 7fbe52ffd640 1 -- 172.21.15.47:0/2204371764 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+746060 (secure 0 0 0) 0x7fbe5800de20 con 0x7fbe5c103230 2024-04-25T21:10:04.119 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:10:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:10:04.119 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 25, 2024-04-25T21:10:04.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.119 INFO:teuthology.orchestra.run.smithi047.stdout: "active_gid": 24811, 2024-04-25T21:10:04.119 INFO:teuthology.orchestra.run.smithi047.stdout: "active_name": "x", 2024-04-25T21:10:04.119 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addrs": { 2024-04-25T21:10:04.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6800", 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 957595556 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6801", 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 957595556 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addr": "172.21.15.47:6801/957595556", 2024-04-25T21:10:04.120 INFO:teuthology.orchestra.run.smithi047.stdout: "active_change": "2024-04-25T20:59:59.008005+0000", 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "available": true, 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "standbys": [ 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 24805, 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "z", 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.122 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.123 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.124 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.125 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.126 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.127 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.128 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-04-25T21:10:04.129 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.130 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.131 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:10:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:10:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:10:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.147 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:10:04.148 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-04-25T21:10:04.149 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.150 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.152 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.153 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.154 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-04-25T21:10:04.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-04-25T21:10:04.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-04-25T21:10:04.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-04-25T21:10:04.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:10:04.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.160 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:10:04.161 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:10:04.162 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:10:04.163 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:10:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:04.165 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:10:04.166 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.168 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.170 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:10:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.172 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.173 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.174 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.175 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.176 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.177 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.178 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.179 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.180 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:10:04.181 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:10:04.182 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.183 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-04-25T21:10:04.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-04-25T21:10:04.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.187 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.188 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.189 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.190 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.191 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.193 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.194 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.195 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.196 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.198 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.199 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.200 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.201 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.202 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.203 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:10:04.204 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.205 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.206 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.207 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.208 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.209 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.210 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.211 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:10:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.218 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.219 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-04-25T21:10:04.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-04-25T21:10:04.222 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.224 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.225 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.226 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.227 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.228 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.229 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.230 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.232 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:10:04.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:10:04.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:04.237 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.239 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.241 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.242 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.243 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.244 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.245 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.246 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.248 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.249 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.250 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.252 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.253 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.254 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.255 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.258 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.259 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.260 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.261 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.262 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.263 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.264 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.265 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.267 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.268 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.269 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.270 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.273 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.274 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.275 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.276 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.277 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.278 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.280 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-04-25T21:10:04.281 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.283 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.284 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.285 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.287 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.288 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.289 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.290 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.291 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.293 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.294 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.295 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.296 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.297 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:10:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-04-25T21:10:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.322 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.323 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.325 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.327 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.328 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.329 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.330 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.331 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.332 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.333 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.334 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.335 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.336 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.338 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.339 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.340 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.341 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.342 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.343 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.344 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.345 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.346 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.347 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-04-25T21:10:04.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.350 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.352 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.353 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.354 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.355 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.356 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.357 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.359 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.362 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.363 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.364 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.365 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.366 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.369 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.370 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.372 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.373 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.374 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-25T21:10:04.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.385 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.386 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.387 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.390 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.392 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.394 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:10:04.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.396 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.398 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.399 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.402 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.403 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.404 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.405 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.406 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.407 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.410 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.411 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.412 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-04-25T21:10:04.413 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.414 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.415 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:04.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-04-25T21:10:04.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-25T21:10:04.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:10:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-04-25T21:10:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-25T21:10:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:10:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.484 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-04-25T21:10:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-04-25T21:10:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-04-25T21:10:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-04-25T21:10:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-25T21:10:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-04-25T21:10:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-04-25T21:10:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-04-25T21:10:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-04-25T21:10:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 24836, 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "y", 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-04-25T21:10:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-04-25T21:10:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-04-25T21:10:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-04-25T21:10:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-04-25T21:10:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-04-25T21:10:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-04-25T21:10:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-25T21:10:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-04-25T21:10:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-04-25T21:10:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-04-25T21:10:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-04-25T21:10:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-04-25T21:10:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-04-25T21:10:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:10:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:10:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:10:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:10:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-04-25T21:10:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-04-25T21:10:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-04-25T21:10:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-04-25T21:10:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:10:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:10:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:10:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:10:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:10:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-04-25T21:10:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-25T21:10:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-04-25T21:10:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-04-25T21:10:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-04-25T21:10:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-04-25T21:10:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-04-25T21:10:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:10:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:10:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:10:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-04-25T21:10:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:10:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-04-25T21:10:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-25T21:10:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-04-25T21:10:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-04-25T21:10:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.892 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.898 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.899 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.900 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-04-25T21:10:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-04-25T21:10:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-25T21:10:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:10:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "modules": [ 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard", 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "iostat", 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "nfs", 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "restful", 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_schedule" 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-04-25T21:10:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-04-25T21:10:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-25T21:10:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-04-25T21:10:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-04-25T21:10:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-04-25T21:10:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-04-25T21:10:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-04-25T21:10:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-04-25T21:10:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-04-25T21:10:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-04-25T21:10:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-04-25T21:10:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:10:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:10:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:10:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:10:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:10:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:10:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:10:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-04-25T21:10:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-04-25T21:10:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-04-25T21:10:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-25T21:10:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:10:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-04-25T21:10:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:10:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-04-25T21:10:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:10:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:10:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:10:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:10:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:10:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-25T21:10:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-25T21:10:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-04-25T21:10:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-04-25T21:10:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-04-25T21:10:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:10:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-25T21:10:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:10:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-04-25T21:10:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-04-25T21:10:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-25T21:10:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-04-25T21:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-04-25T21:10:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:10:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-04-25T21:10:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-04-25T21:10:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:10:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:10:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:10:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-04-25T21:10:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-04-25T21:10:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-04-25T21:10:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-04-25T21:10:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.300 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-04-25T21:10:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-04-25T21:10:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-04-25T21:10:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:10:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-04-25T21:10:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-04-25T21:10:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:10:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-25T21:10:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-25T21:10:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "services": { 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard": "https://172.21.15.47:8443/" 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "always_on_modules": { 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus": [ 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-25T21:10:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific": [ 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-25T21:10:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy": [ 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "reef": [ 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-25T21:10:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "squid": [ 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-25T21:10:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "last_failure_osd_epoch": 92, 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "active_clients": [ 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-25T21:10:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1344209874 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "libcephsqlite", 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3898390877 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-04-25T21:10:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4045979974 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3432188029 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1223447400 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-25T21:10:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-25T21:10:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-25T21:10:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2494829252 2024-04-25T21:10:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-25T21:10:05.387 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:10:05.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:04 smithi092 ceph-mon[32524]: pgmap v311: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:05.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:04 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1304201217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:10:05.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:04 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2121750230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:10:05.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:04 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/2204371764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:10:05.389 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:04 smithi186 ceph-mon[32529]: pgmap v311: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:05.389 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:04 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1304201217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:10:05.389 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:04 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2121750230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:10:05.389 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:04 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/2204371764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:10:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbe480a3d20 msgr2=0x7fbe480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbe480a3d20 0x7fbe480a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c10e270 tx=0x7fbe4c005f90 comp rx=0 tx=0).stop 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c103230 msgr2=0x7fbe5c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c103230 0x7fbe5c198650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c920 tx=0x7fbe5800cde0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 shutdown_connections 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fbe480a3d20 0x7fbe480a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5c10c570 0x7fbe5c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5c103b50 0x7fbe5c198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 --2- 172.21.15.47:0/2204371764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe5c103230 0x7fbe5c198650 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 >> 172.21.15.47:0/2204371764 conn(0x7fbe5c077fb0 msgr2=0x7fbe5c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 shutdown_connections 2024-04-25T21:10:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:04.114+0000 7fbe64108640 1 -- 172.21.15.47:0/2204371764 wait complete. 2024-04-25T21:10:05.432 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T21:10:05.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T21:10:05.432 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:04 smithi047 ceph-mon[29195]: pgmap v311: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:05.433 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:04 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1304201217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:10:05.433 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:04 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2121750230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:10:05.433 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:04 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/2204371764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:10:05.693 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:06.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:06 smithi047 ceph-mon[29195]: pgmap v312: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-04-25T21:10:06.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:06 smithi186 ceph-mon[32529]: pgmap v312: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-04-25T21:10:06.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:06 smithi092 ceph-mon[32524]: pgmap v312: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-04-25T21:10:07.504 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:10:07.504 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:07.504 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 229, 2024-04-25T21:10:07.504 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 219, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1453360, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 134464, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.041752002, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.013549214, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.030709794, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.004244423, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 218, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.009970505, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000045736 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1453360, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 134464 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 167, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 185600, 2024-04-25T21:10:07.506 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.013116151, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.008953965, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001778661, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 164, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.015584970, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000095030 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:07.507 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 185600 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 4, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 4, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 3040, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 448, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.000650926, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000041132, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000589307, 2024-04-25T21:10:07.508 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000753820, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000251273 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.509 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:07.510 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:07.511 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:07.513 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:07.513 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:07.514 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148754, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 31, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-04-25T21:10:07.516 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 231, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1421044, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 231, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-04-25T21:10:07.517 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 231, 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1421044, 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:07.518 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:10:08.276 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T21:10:08.277 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T21:10:08.277 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-25T21:10:08.277 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-25T21:10:08.277 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T21:10:08.277 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T21:10:08.277 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T21:10:08.277 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-25T21:10:08.277 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T21:10:08.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T21:10:08.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:08 smithi047 ceph-mon[29195]: pgmap v313: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:10:08.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:08 smithi047 ceph-mon[29195]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:10:08.611 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:08 smithi186 ceph-mon[32529]: pgmap v313: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:10:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:08 smithi186 ceph-mon[32529]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:10:08.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:08 smithi092 ceph-mon[32524]: pgmap v313: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:10:08.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:08 smithi092 ceph-mon[32524]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:10:10.354 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:10:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-04-25T21:10:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-04-25T21:10:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-04-25T21:10:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-04-25T21:10:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.001361915, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000291590, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 172, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186480, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.014663598, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.010193190, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001811774, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 169, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.009852404, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000058298 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186480 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 239, 2024-04-25T21:10:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 227, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1458032, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 138736, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.048026014, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.015037590, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.037523852, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.004594751, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 227, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.011410187, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000050265 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1458032, 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 138736 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 149762, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 32, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 237, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1421824, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 237, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 237, 2024-04-25T21:10:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1421824, 2024-04-25T21:10:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-04-25T21:10:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:10.365 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:10:10.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:10 smithi047 ceph-mon[29195]: pgmap v315: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T21:10:10.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:10 smithi186 ceph-mon[32529]: pgmap v315: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T21:10:10.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:10 smithi092 ceph-mon[32524]: pgmap v315: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T21:10:11.313 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T21:10:11.313 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T21:10:11.313 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T21:10:11.313 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T21:10:11.313 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T21:10:11.313 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T21:10:11.313 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T21:10:11.313 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-25T21:10:11.313 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T21:10:11.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T21:10:11.569 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:12.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:12 smithi047 ceph-mon[29195]: pgmap v316: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T21:10:12.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:12 smithi047 ceph-mon[29195]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:10:12.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:12 smithi186 ceph-mon[32529]: pgmap v316: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T21:10:12.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:12 smithi186 ceph-mon[32529]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:10:12.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:12 smithi092 ceph-mon[32524]: pgmap v316: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-04-25T21:10:12.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:12 smithi092 ceph-mon[32524]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:10:13.116 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 4, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 4, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 3040, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 448, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.001660753, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000040941, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000550671, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3, 2024-04-25T21:10:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000749512, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000249837 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 168, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:10:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 185776, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.014112641, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.009584565, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.002041879, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 165, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.014515893, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000087975 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 185776 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 231, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 221, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1453904, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 135744, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.045132897, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.013955926, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.034483649, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.004601868, 2024-04-25T21:10:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 220, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.009795615, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000044525 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1453904, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 135744 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-04-25T21:10:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148754, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 31, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-04-25T21:10:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 233, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1421304, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 233, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 233, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1421304, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:13.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:13.127 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:10:13.917 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T21:10:13.917 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T21:10:13.917 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T21:10:13.917 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T21:10:13.917 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T21:10:13.917 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T21:10:13.917 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T21:10:13.917 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-25T21:10:13.917 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T21:10:13.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:14.345 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:14 smithi092 ceph-mon[32524]: pgmap v318: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:10:14.346 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:14 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:10:14.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:14 smithi047 ceph-mon[29195]: pgmap v318: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:10:14.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:14 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:10:14.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:14 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:14.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:14 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:14 smithi186 ceph-mon[32529]: pgmap v318: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T21:10:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:14 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:10:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:14 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:14.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:14 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:14.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:14 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:14.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:14 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.001403190, 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T21:10:15.940 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000272988, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.941 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 241, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 229, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1458576, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 140032, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.045690105, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.014532351, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.034092934, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.004620516, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 229, 2024-04-25T21:10:15.942 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.011831611, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000051666 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1458576, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 140032 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 174, 2024-04-25T21:10:15.943 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186832, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.013407560, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.009278505, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001985631, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 171, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.016216681, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000094834 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:15.944 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186832 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:15.945 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:15.946 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:15.947 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148794, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 32, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:15.948 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:15.949 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 239, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1422084, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 239, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 239, 2024-04-25T21:10:15.950 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1422084, 2024-04-25T21:10:15.951 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-04-25T21:10:15.951 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-25T21:10:15.951 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-25T21:10:15.951 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-25T21:10:15.951 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:15.951 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-25T21:10:15.951 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-25T21:10:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: pgmap v319: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:10:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: Unable to set osd_memory_target on smithi186 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T21:10:16.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:16 smithi186 ceph-mon[32529]: Unable to set osd_memory_target on smithi092 to 184781721: error parsing value: Value '184781721' is below minimum 939524096 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: pgmap v319: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-25T21:10:16.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-04-25T21:10:16.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: Unable to set osd_memory_target on smithi186 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T21:10:16.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:16 smithi092 ceph-mon[32524]: Unable to set osd_memory_target on smithi092 to 184781721: error parsing value: Value '184781721' is below minimum 939524096 2024-04-25T21:10:16.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: pgmap v319: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-25T21:10:16.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-04-25T21:10:16.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: Unable to set osd_memory_target on smithi186 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T21:10:16.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:16 smithi047 ceph-mon[29195]: Unable to set osd_memory_target on smithi092 to 184781721: error parsing value: Value '184781721' is below minimum 939524096 2024-04-25T21:10:16.585 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T21:10:16.585 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T21:10:16.585 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T21:10:16.585 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T21:10:16.585 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T21:10:16.585 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T21:10:16.585 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T21:10:16.585 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-25T21:10:16.585 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T21:10:16.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T21:10:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2318, 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1660, 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 913331, 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 5168888, 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 23, 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.246862608, 2024-04-25T21:10:18.702 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.081518834, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.081576516, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.074821235, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1661, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.566151705, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000340849 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2220, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000412871, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000185 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.703 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 338128, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 89696 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 4097, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2602, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3763080, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 2488046, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 55, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 7, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.264853929, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.104600250, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.152218244, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.033235905, 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:18.704 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2602, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.464071430, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000178351 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5554, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000825028, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000148 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 44272, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 467552 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 3292, 2024-04-25T21:10:18.705 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 3709, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2583236, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 1999446, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 17, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 8, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.280470569, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.087609094, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.149304030, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.063493324, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3703, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.654012748, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000176616 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.706 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5196, 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000820333, 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000157 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1664112, 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 181808 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-04-25T21:10:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 801, 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.019330687, 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001272572 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 50, 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001617608, 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000032352 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.708 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 994, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 937, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 937, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 61.008156278, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.065110092 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 57, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 176, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 1, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-04-25T21:10:18.709 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 29, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 7, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 12, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 10, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 17, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 1002, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 3, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 3, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 3, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 1314, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 894, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 54, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 31, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:10:18.710 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 2, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 20, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 882, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 1, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 4, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 4, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 234, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 6, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:10:18.711 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 2, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 247, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 515, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 1407, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 1135, 2024-04-25T21:10:18.712 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 19, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 97 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 11, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 374, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 202, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 205, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 43, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 1, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 21, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 17, 2024-04-25T21:10:18.713 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 3, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 2, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 376, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 20, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 1, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-04-25T21:10:18.714 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 1453, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 1404, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 1404, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 49, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 6, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 24, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 23, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 23, 2024-04-25T21:10:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 1, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 1, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 1, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 8306063, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 8363426, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 407, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 21.154358395, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.051976310 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-04-25T21:10:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 27, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 685, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 658, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 26, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 253, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 227, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 29, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 1054, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 1025, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 3, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 465, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 462, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 65180, 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 231800 2024-04-25T21:10:18.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 1072, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 44, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 994, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 40, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 92, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 407, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 60.548027220, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.148766651 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 104, 2024-04-25T21:10:18.718 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.034857933, 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000335172 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3, 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.004201677, 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001400559 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-04-25T21:10:18.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 18, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.041906809, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002328156 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000317152, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000317152 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.720 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.722 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-04-25T21:10:18.722 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 10, 2024-04-25T21:10:18.722 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001645540, 2024-04-25T21:10:18.722 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000164554 2024-04-25T21:10:18.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.722 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-04-25T21:10:18.722 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 45, 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.126123065, 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002802734 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 10, 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.054413335, 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005441333 2024-04-25T21:10:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 289, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.055197781, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000190995 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.011993351, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002998337 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 20, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.058597512, 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002929875 2024-04-25T21:10:18.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.008053396, 2024-04-25T21:10:18.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.008053396 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.018354240, 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004588560 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-04-25T21:10:18.726 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001536078, 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001536078 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-04-25T21:10:18.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.022498831, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002812353 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000647768, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000647768 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.728 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 11, 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.019784590, 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001798599 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-04-25T21:10:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 6, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 4, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 29, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 14, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 588, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 360, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 360, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:18.730 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:18.731 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-04-25T21:10:18.732 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 311793, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 108, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 88, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 1, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8136, 2024-04-25T21:10:18.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 170, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 149154, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 861, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-04-25T21:10:18.734 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 1715, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 4972233, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 32, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 1715, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1715, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 27.704230698, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.016154070 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1715, 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2897 2024-04-25T21:10:18.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 1715, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 4, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 1711, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 114, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 429, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 4, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 411, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 101, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 1, 2024-04-25T21:10:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 529, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 859, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-04-25T21:10:18.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-04-25T21:10:18.738 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 100, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 11, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 420, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 29 2024-04-25T21:10:18.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 420 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-04-25T21:10:18.740 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 10, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 17, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 17, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 9703, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 6317086, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 9703, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:18.741 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 9703, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 6317086, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-04-25T21:10:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1715, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 4973950, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1715, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 4973950, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-04-25T21:10:18.743 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1715, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 1715, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1715, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1715, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 17, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1717, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 17, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 4, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1717, 2024-04-25T21:10:18.745 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x5647072b8500": { 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 1453, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 4169122, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 1453, 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 407, 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 4169122, 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:18.747 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: pgmap v320: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:10:19.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:18 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: pgmap v320: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:10:19.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:10:19.100 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:18 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: pgmap v320: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:10:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:10:19.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:18 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' 2024-04-25T21:10:19.563 INFO:tasks.check_counter:Daemon mds.b mds.exported=20 2024-04-25T21:10:19.564 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-04-25T21:10:19.564 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=11 2024-04-25T21:10:19.564 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=374 2024-04-25T21:10:19.564 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T21:10:19.564 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T21:10:19.564 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T21:10:19.564 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-04-25T21:10:19.564 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T21:10:19.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T21:10:19.817 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:10:19.847 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:19 smithi092 ceph-mon[32524]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-25T21:10:19.847 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:19 smithi092 ceph-mon[32524]: Unable to set osd_memory_target on smithi047 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T21:10:20.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:19 smithi047 ceph-mon[29195]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-25T21:10:20.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:19 smithi047 ceph-mon[29195]: Unable to set osd_memory_target on smithi047 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T21:10:20.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:19 smithi186 ceph-mon[32529]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-25T21:10:20.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:19 smithi186 ceph-mon[32529]: Unable to set osd_memory_target on smithi047 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T21:10:20.983 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:20 smithi092 ceph-mon[32524]: pgmap v321: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:10:21.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:20 smithi047 ceph-mon[29195]: pgmap v321: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:10:21.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:20 smithi186 ceph-mon[32529]: pgmap v321: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 68259, 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 120101, 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 19357384, 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 118014619, 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 16, 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 4.204969242, 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 1.722332440, 2024-04-25T21:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 2.521305714, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.679120680, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 120099, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.891710416, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000024077 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 106788, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.016646184, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000155 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 736 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.500 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 19161, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 19346, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 4788022, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 60059521, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 12, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 7, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 1.298463451, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.414890604, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.297117777, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.498426140, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 19343, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.639568885, 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000033064 2024-04-25T21:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 32, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000008130, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000254 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1888, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 465696 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 20902, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 23763, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 7943242, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 26752315, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 9, 2024-04-25T21:10:21.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 8, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 1.528181504, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.591483425, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.462170614, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.429315254, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 23763, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.748046490, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000031479 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 9159, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001582447, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000172 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1552208, 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 148624 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 86553, 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 28.085734984, 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000324491 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7758, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.658213811, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000342641 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 22593, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 22539, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 22539, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 63.007602133, 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002795492 2024-04-25T21:10:21.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 54, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 184, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 21, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 11139, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 32, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 9313, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 1794, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 11091, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 19, 2024-04-25T21:10:21.506 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 1, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 1, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 5, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 26509, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 26031, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 52, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 22, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 2, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 14, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 21, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 905, 2024-04-25T21:10:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 4278, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 18644445, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 3, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-04-25T21:10:21.508 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 6, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 8243, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 18262, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 7863, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 15237, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 13766, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 220, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 14747 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 379, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 17, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 11, 2024-04-25T21:10:21.509 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 31, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 462, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 60, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 5140, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 5080, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-04-25T21:10:21.510 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 2, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 18, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 2, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 21, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 49015, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 49015, 2024-04-25T21:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 73, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 73, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 5, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 156028897, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 24591, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 50.745229129, 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002063569 2024-04-25T21:10:21.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 6359, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 11452, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 5093, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 231, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 262, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 31, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 11139, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 16623, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 5484, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 1, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 5298, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 5297, 2024-04-25T21:10:21.513 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 118128, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 223608 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 41091, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 17, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 22593, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 22, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 72, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4691, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 17.030433278, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003630448 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7, 2024-04-25T21:10:21.514 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.024339251, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003477035 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3052, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.548524607, 2024-04-25T21:10:21.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000507380 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000134905, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000134905 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 27, 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.553280050, 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.020491853 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.518 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 128, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.064892936, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000506976 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 461, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.692764560, 2024-04-25T21:10:21.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003671940 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 72, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.159636531, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002217174 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 65, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3.026023426, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.046554206 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8997, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 30.246711562, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003361866 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002744413, 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000343051 2024-04-25T21:10:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003261971, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000652394 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.523 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5025, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 8.654854643, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001722359 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 3, 2024-04-25T21:10:21.524 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 11623, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 5811, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 587, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:21.525 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:21.526 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 956417, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 5066, 2024-04-25T21:10:21.527 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8136, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 170, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 22798882, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 209203, 2024-04-25T21:10:21.528 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 66935, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 155140575, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 66935, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 66935, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 185.215843600, 2024-04-25T21:10:21.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002767100 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 66935, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 67222 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 66935, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 54, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 66881, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 54, 2024-04-25T21:10:21.530 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 24645, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 46, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 5, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 5176, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-04-25T21:10:21.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 36717, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-04-25T21:10:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 100, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 11, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 509, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-04-25T21:10:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 40 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 20 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 34, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-04-25T21:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 10, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 5112, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 5080, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 108318, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 22321279, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 108318, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 108318, 2024-04-25T21:10:21.535 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 22321279, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 66935, 2024-04-25T21:10:21.536 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 155763935, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 66935, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 155763935, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 66935, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 66935, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 66935, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 66935, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 5080, 2024-04-25T21:10:21.538 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 623360, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 5080, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 54, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 623360, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x5583f68ea500": { 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 49015, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 151834593, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 49015, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 24591, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 151834593, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:21.541 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-25T21:10:22.258 INFO:tasks.check_counter:Daemon mds.e mds.exported=2 2024-04-25T21:10:22.258 INFO:tasks.check_counter:Daemon mds.e mds.imported=21 2024-04-25T21:10:22.258 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=379 2024-04-25T21:10:22.258 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=17 2024-04-25T21:10:22.258 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=3 2024-04-25T21:10:22.258 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:10:22.258 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:10:22.258 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=21 2024-04-25T21:10:22.258 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T21:10:22.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T21:10:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:10:22.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:22 smithi092 ceph-mon[32524]: pgmap v322: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:10:23.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:22 smithi047 ceph-mon[29195]: pgmap v322: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:10:23.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:22 smithi186 ceph-mon[32529]: pgmap v322: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:10:24.192 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-25T21:10:24.192 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 3313, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2042, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3453706, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 758013, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 11, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.186885337, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.084811280, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.122189024, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.005324093, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2041, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.069590841, 2024-04-25T21:10:24.193 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000034096 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5117, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000788143, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000154 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 352, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 353, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1565854, 2024-04-25T21:10:24.194 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 243506, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 15, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.054517005, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.016777387, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.040664386, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.006002132, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 352, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.026954004, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000076573 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 71, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000015257, 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000214 2024-04-25T21:10:24.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1557984, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 159600 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 183, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 403, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 49332, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 542115, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 10, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.027275921, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.012328003, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.013868153, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000462383, 2024-04-25T21:10:24.196 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 400, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.031364031, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000078410 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 353, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000068797, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000194 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 460368 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-04-25T21:10:24.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 108, 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.023126356, 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000214132 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 35, 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001233139, 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000035232 2024-04-25T21:10:24.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 147, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 41, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 41, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.049375479, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001204279 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 106, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 11, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-04-25T21:10:24.199 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 18, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 2, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 5, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 11, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 14, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 152, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 1, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 1, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 4, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 475, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 276, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 103, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 59, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:10:24.200 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 2, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 27, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 2, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 14, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 4278, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 18644445, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 3, 2024-04-25T21:10:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 19, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:10:24.202 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 8, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 2, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 5, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 28 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 3, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 486, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 271, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 273, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 4, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 1, 2024-04-25T21:10:24.203 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 11, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 10, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 2, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 18, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:10:24.204 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 2, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 53, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 53, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 2, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-04-25T21:10:24.205 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 2, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 4299836, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 29, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.061462907, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002119410 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-04-25T21:10:24.206 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 23, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 116, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 93, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 22, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 250, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 228, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 18, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 184, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 166, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 1, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 23, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 22, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 54444, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 223608 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.207 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 174, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 5, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 147, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 20, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 44, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006658165, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003329082 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001000228, 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000250057 2024-04-25T21:10:24.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 18, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.007702508, 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000427917 2024-04-25T21:10:24.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-04-25T21:10:24.210 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-04-25T21:10:24.211 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002826011, 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002826011 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.212 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000195705, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000195705 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-04-25T21:10:24.213 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001708961, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000427240 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-04-25T21:10:24.214 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.005714906, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005714906 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.215 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.216 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 10, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.023568995, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002356899 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-04-25T21:10:24.218 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 3, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 23, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 11, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 542, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:24.220 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:24.221 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 297340, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 110, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8136, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 170, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:24.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 99644, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 540, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 107, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 120276, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-04-25T21:10:24.223 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 107, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 107, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.590983569, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005523210 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 107, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 140 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 107, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 3, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 104, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-04-25T21:10:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 3, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 32, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 17, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 22, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:10:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-04-25T21:10:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 100, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 11, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 11, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 11, 2024-04-25T21:10:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 30, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 6 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 20 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 7, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 7, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 10, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 10, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 3844, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 4703772, 2024-04-25T21:10:24.229 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 3844, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3844, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 4703772, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-04-25T21:10:24.230 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 107, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 121574, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 107, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 121574, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 107, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 107, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 107, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 107, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.232 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 10, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1298, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 10, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1298, 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:24.233 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x562f3f408500": { 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 53, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 105532, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 53, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:24.234 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 29, 2024-04-25T21:10:24.235 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 105532, 2024-04-25T21:10:24.235 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:24.235 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:24.235 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:24.235 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:24.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:24.235 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-25T21:10:24.947 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:24 smithi092 ceph-mon[32524]: pgmap v323: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:10:24.976 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-04-25T21:10:24.977 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-04-25T21:10:24.977 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=3 2024-04-25T21:10:24.977 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=486 2024-04-25T21:10:24.977 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=3 2024-04-25T21:10:24.977 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:10:24.977 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:10:24.977 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-25T21:10:24.977 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T21:10:24.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T21:10:25.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:24 smithi047 ceph-mon[29195]: pgmap v323: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:10:25.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:24 smithi186 ceph-mon[32529]: pgmap v323: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-25T21:10:25.220 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:10:26.869 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-25T21:10:26.869 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:26.869 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 0, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 0, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 0, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.001086285, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.000321420, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:26.870 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:26.871 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 245, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 233, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1459664, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 142592, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.046527638, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.014451102, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.034313800, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.005732981, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 233, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.012514698, 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000053711 2024-04-25T21:10:26.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1459664, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 142592 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 177, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:10:26.873 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 187360, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.011311512, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.007484678, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001482593, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 174, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.018594370, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000106864 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 187360 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:26.875 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:26.876 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:26.877 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 149735, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 30, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-04-25T21:10:26.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 928, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:26.879 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 243, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1422604, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 243, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 243, 2024-04-25T21:10:26.880 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1422604, 2024-04-25T21:10:26.881 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-04-25T21:10:26.881 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-04-25T21:10:26.881 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-04-25T21:10:26.881 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-04-25T21:10:26.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:26.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-25T21:10:26.881 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-25T21:10:26.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:26 smithi092 ceph-mon[32524]: pgmap v324: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:10:27.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:26 smithi186 ceph-mon[32529]: pgmap v324: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:10:27.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:26 smithi047 ceph-mon[29195]: pgmap v324: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:10:27.543 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T21:10:27.543 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T21:10:27.543 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T21:10:27.543 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T21:10:27.543 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T21:10:27.543 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T21:10:27.543 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T21:10:27.543 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-25T21:10:27.543 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T21:10:27.544 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T21:10:27.791 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:10:29.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:28 smithi092 ceph-mon[32524]: pgmap v325: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:29.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:28 smithi186 ceph-mon[32529]: pgmap v325: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:29.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:28 smithi047 ceph-mon[29195]: pgmap v325: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:29.470 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-25T21:10:29.470 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:29.470 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2329, 2024-04-25T21:10:29.470 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2331, 2024-04-25T21:10:29.470 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 763889, 2024-04-25T21:10:29.470 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 740629, 2024-04-25T21:10:29.470 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.198914308, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.064126812, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.052664284, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.067161296, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2329, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.083974745, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000036056 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 269, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 245, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1537116, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 143050, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 3, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.047431487, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.014952264, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.034678124, 2024-04-25T21:10:29.472 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006185799, 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 244, 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012970323, 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000053157 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1536560, 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 142352 2024-04-25T21:10:29.473 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 7, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 176, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3628, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 461310, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.010920193, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.006724522, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.002362156, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000030604, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 174, 2024-04-25T21:10:29.474 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.014373589, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000082606 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 460992 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.475 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3954, 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.104641658, 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000026464 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.476 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-25T21:10:29.477 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 12, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 12, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 12, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2024-04-25T21:10:29.478 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-25T21:10:29.479 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:10:29.480 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-25T21:10:29.481 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-25T21:10:29.482 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-25T21:10:29.483 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 53, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-25T21:10:29.484 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4299836, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 4298221, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 53 2024-04-25T21:10:29.485 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 5, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 45, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 40, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 3, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 19, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 16, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 12, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 55, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 43, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-25T21:10:29.486 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 50176, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215416 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.487 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.489 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-25T21:10:29.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-25T21:10:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.492 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-25T21:10:29.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.494 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.495 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.497 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.498 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 3, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 1776, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-25T21:10:29.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:29.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:29.501 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 182158, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 50, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8136, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 170, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 25982, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 210, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-25T21:10:29.503 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2332, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2332, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2332, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.163979081, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000499133 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2332, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2333 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.504 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2332, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2332, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1503, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 827, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-25T21:10:29.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-25T21:10:29.506 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 100, 2024-04-25T21:10:29.507 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 11, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 4, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 4, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-25T21:10:29.508 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-25T21:10:29.509 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2601, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 2053834, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2601, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2601, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2053834, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.510 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2332, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 105537, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2332, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 105537, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.511 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2332, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2332, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2332, 2024-04-25T21:10:29.512 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2332, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:29.513 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x5626e3e566e0": { 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:29.514 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:29.515 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-25T21:10:30.225 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T21:10:30.225 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T21:10:30.225 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T21:10:30.225 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T21:10:30.225 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T21:10:30.225 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:10:30.225 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:10:30.225 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-25T21:10:30.226 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T21:10:30.226 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T21:10:30.467 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:10:30.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:30 smithi186 ceph-mon[32529]: pgmap v326: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:31.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:30 smithi092 ceph-mon[32524]: pgmap v326: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:31.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:30 smithi047 ceph-mon[29195]: pgmap v326: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:32.073 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-25T21:10:32.073 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:32.073 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 519, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 519, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 42090091, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 164282, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 7, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.068948613, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.014963307, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.035132012, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.013499440, 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:32.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 518, 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.023097319, 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000044589 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.075 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1130, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1303, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 72430593, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 822837, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 8, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.121490954, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.033461765, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.049823061, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.029765899, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1300, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.085173509, 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000065518 2024-04-25T21:10:32.076 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 464128 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1243, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1219, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 39604978, 2024-04-25T21:10:32.077 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 453364, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.171169063, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.047359832, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.082245193, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.040738584, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1218, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.058275971, 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000047845 2024-04-25T21:10:32.078 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1537104, 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 143632 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.079 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4347, 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.120041202, 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000027614 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-25T21:10:32.081 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 16447, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 193, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 16254, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 16171, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-25T21:10:32.082 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-25T21:10:32.083 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:10:32.084 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-25T21:10:32.085 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:10:32.086 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-04-25T21:10:32.087 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 49015, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 73, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 5, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 156028897, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 156025346, 2024-04-25T21:10:32.088 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 49015 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 6354, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 11451, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 5097, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 225, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 259, 2024-04-25T21:10:32.089 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 34, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 16447, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 16477, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 30, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 86604, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215416 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-04-25T21:10:32.090 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.091 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.092 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.093 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.094 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.095 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.096 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.097 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.098 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.099 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.100 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-25T21:10:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 3, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 1778, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:32.104 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 462179, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 6528, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8136, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 170, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 26556913, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 225405, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-25T21:10:32.106 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2617, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2617, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2617, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.657970185, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000633538 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2617, 2024-04-25T21:10:32.107 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2618 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2617, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2617, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1603, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 1012, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-25T21:10:32.108 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-25T21:10:32.109 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-25T21:10:32.110 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 100, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 11, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 9, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 9, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-25T21:10:32.111 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-25T21:10:32.112 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-25T21:10:32.113 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2888, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 153848929, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2888, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2888, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 153848929, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.114 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2617, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 151852220, 2024-04-25T21:10:32.115 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2617, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 151852220, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:32.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2617, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2617, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2617, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2617, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:32.117 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.118 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x557cab2c86e0": { 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-25T21:10:32.119 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:32.120 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:32.120 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:32.120 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:32.120 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.120 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:32.120 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-25T21:10:32.826 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:32 smithi186 ceph-mon[32529]: pgmap v327: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:32.854 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T21:10:32.854 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T21:10:32.854 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T21:10:32.854 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T21:10:32.855 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T21:10:32.855 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:10:32.855 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:10:32.855 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-25T21:10:32.855 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T21:10:32.855 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T21:10:33.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:32 smithi092 ceph-mon[32524]: pgmap v327: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:33.097 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:10:33.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:32 smithi047 ceph-mon[29195]: pgmap v327: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:34.670 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-25T21:10:34.670 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:34.670 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1712, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1714, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 4676225, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 544426, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.157712263, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.049836086, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.043681559, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.049388424, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1712, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.062487402, 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000036499 2024-04-25T21:10:34.671 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:34.672 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 10, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 179, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 5016, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 463946, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 7, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.011164305, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.006736307, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.002547887, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000133858, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 177, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.016542235, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000093458 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.673 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 462608 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1067, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1043, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1733345, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 396595, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 3, 2024-04-25T21:10:34.674 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.118086411, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.041068368, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.048001730, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.030129115, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1042, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.039357121, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000037770 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.675 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1536720, 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 142512 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4149, 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.113044555, 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000027246 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-25T21:10:34.676 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.677 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 42, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 21, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 21, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 11, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-25T21:10:34.678 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 2, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-25T21:10:34.679 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 4, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 234, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 6, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:10:34.680 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-25T21:10:34.681 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:10:34.682 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 1453, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-04-25T21:10:34.683 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 24, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 8306063, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 8363426, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 8363314, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 1453 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-25T21:10:34.684 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 17, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 638, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 621, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 12, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 58, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 46, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 42, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 659, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 617, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 54892, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215416 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-25T21:10:34.685 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.686 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.687 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-25T21:10:34.688 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.689 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-25T21:10:34.690 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.690 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.690 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.690 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.690 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-25T21:10:34.690 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.690 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.690 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.691 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.692 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.693 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.694 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.695 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.696 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 6, 2024-04-25T21:10:34.697 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 4, 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 5, 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 3506, 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:34.698 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:34.701 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:34.703 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 190383, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 58, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:34.704 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8136, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 170, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 106055, 2024-04-25T21:10:34.705 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 592, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2515, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 26, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 4, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2515, 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-25T21:10:34.706 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2515, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.116979923, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000444127 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2515, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2516 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2515, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2515, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1603, 2024-04-25T21:10:34.707 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 909, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 2, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-25T21:10:34.708 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-25T21:10:34.709 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 100, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 11, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 5, 2024-04-25T21:10:34.710 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 5, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-25T21:10:34.711 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:34.712 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2785, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 6147181, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2785, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2785, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 6147181, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-25T21:10:34.713 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2515, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 4169132, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2515, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 4169132, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.714 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2515, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2515, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2515, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2515, 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:34.715 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55ab2974e6e0": { 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:34.717 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:34.718 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-25T21:10:35.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:34 smithi092 ceph-mon[32524]: pgmap v328: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:35.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:34 smithi047 ceph-mon[29195]: pgmap v328: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:35.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:34 smithi186 ceph-mon[32529]: pgmap v328: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T21:10:35.468 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T21:10:35.468 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T21:10:35.468 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T21:10:35.468 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-25T21:10:35.468 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T21:10:35.468 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:10:35.468 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:10:35.468 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-25T21:10:35.468 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T21:10:35.469 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T21:10:35.707 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:10:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:36 smithi186 ceph-mon[32529]: pgmap v329: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:37.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:36 smithi092 ceph-mon[32524]: pgmap v329: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:37.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:36 smithi047 ceph-mon[29195]: pgmap v329: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:37.218 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-25T21:10:37.218 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:10:37.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 4, 2024-04-25T21:10:37.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 4, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3040, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 448, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.001572978, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000044157, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000729014, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000844597, 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000281532 2024-04-25T21:10:37.219 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 244, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 234, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1457440, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 144064, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2024-04-25T21:10:37.220 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.046725452, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.014499052, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.033283605, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.005891069, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 233, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012785260, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000054872 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1457440, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 144064 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 179, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 187712, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.012106722, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.008106808, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001532091, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:10:37.222 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 176, 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.017764480, 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000100934 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 187712 2024-04-25T21:10:37.223 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:10:37.224 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:10:37.225 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148748, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 32, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-25T21:10:37.226 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-25T21:10:37.227 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 246, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1422994, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 246, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 246, 2024-04-25T21:10:37.228 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1422994, 2024-04-25T21:10:37.229 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-25T21:10:37.229 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-25T21:10:37.229 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-25T21:10:37.229 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-25T21:10:37.229 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:37.229 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-25T21:10:37.229 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-25T21:10:37.867 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T21:10:37.867 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T21:10:37.867 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T21:10:37.867 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T21:10:37.867 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T21:10:37.867 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T21:10:37.867 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T21:10:37.867 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-25T21:10:37.868 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T21:10:37.878 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T21:10:37.878 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T21:10:37.878 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:10:37.899 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T21:10:37.899 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:10:37.959 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-25T21:10:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:38.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:38 smithi047 ceph-mon[29195]: pgmap v330: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:39.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:38 smithi092 ceph-mon[32524]: pgmap v330: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:38 smithi186 ceph-mon[32529]: pgmap v330: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 -- 172.21.15.47:0/3798705978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554106ec0 msgr2=0x7f755410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 --2- 172.21.15.47:0/3798705978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554106ec0 0x7f755410ea90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f754c00b0d0 tx=0x7f754c00b3e0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 -- 172.21.15.47:0/3798705978 shutdown_connections 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 --2- 172.21.15.47:0/3798705978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554106ec0 0x7f755410ea90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 --2- 172.21.15.47:0/3798705978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7554102300 0x7f75541026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 --2- 172.21.15.47:0/3798705978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7554102cb0 0x7f75541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 -- 172.21.15.47:0/3798705978 >> 172.21.15.47:0/3798705978 conn(0x7f7554077f40 msgr2=0x7f75540ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 -- 172.21.15.47:0/3798705978 shutdown_connections 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.115+0000 7f755b454700 1 -- 172.21.15.47:0/3798705978 wait complete. 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f755b454700 1 Processor -- start 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f755b454700 1 -- start start 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f755b454700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7554102300 0x7f75541992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f755b454700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554102cb0 0x7f75541997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f755b454700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7554106ec0 0x7f755419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f755b454700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7554111d50 con 0x7f7554102cb0 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f755b454700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7554111bd0 con 0x7f7554102300 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f755b454700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7554111ed0 con 0x7f7554106ec0 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f75589ef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554102cb0 0x7f75541997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f75589ef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554102cb0 0x7f75541997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57820/0 (socket says 172.21.15.47:57820) 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f75589ef700 1 -- 172.21.15.47:0/732729261 learned_addr learned my addr 172.21.15.47:0/732729261 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f75591f0700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7554102300 0x7f75541992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.116+0000 7f75599f1700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7554106ec0 0x7f755419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f75589ef700 1 -- 172.21.15.47:0/732729261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7554106ec0 msgr2=0x7f755419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:40.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f75589ef700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7554106ec0 0x7f755419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f75589ef700 1 -- 172.21.15.47:0/732729261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7554102300 msgr2=0x7f75541992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f75589ef700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7554102300 0x7f75541992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f75589ef700 1 -- 172.21.15.47:0/732729261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755419e2b0 con 0x7f7554102cb0 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f75591f0700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7554102300 0x7f75541992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f75589ef700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554102cb0 0x7f75541997e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f754400d980 tx=0x7f754400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f754a7fc700 1 -- 172.21.15.47:0/732729261 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544017070 con 0x7f7554102cb0 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f754a7fc700 1 -- 172.21.15.47:0/732729261 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7544009de0 con 0x7f7554102cb0 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f754a7fc700 1 -- 172.21.15.47:0/732729261 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754400f3c0 con 0x7f7554102cb0 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755419e5a0 con 0x7f7554102cb0 2024-04-25T21:10:40.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.117+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541a5e20 con 0x7f7554102cb0 2024-04-25T21:10:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.118+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7554069730 con 0x7f7554102cb0 2024-04-25T21:10:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.119+0000 7f754a7fc700 1 -- 172.21.15.47:0/732729261 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7544009930 con 0x7f7554102cb0 2024-04-25T21:10:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.120+0000 7f754a7fc700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f75400a3e30 0x7f75400a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.120+0000 7f754a7fc700 1 -- 172.21.15.47:0/732729261 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7f75440d0270 con 0x7f7554102cb0 2024-04-25T21:10:40.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.120+0000 7f75591f0700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f75400a3e30 0x7f75400a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.120+0000 7f75591f0700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f75400a3e30 0x7f75400a62f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7550005fd0 tx=0x7f7550005e30 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:40.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.124+0000 7f754a7fc700 1 -- 172.21.15.47:0/732729261 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f754409c910 con 0x7f7554102cb0 2024-04-25T21:10:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.346+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f755406af90 con 0x7f7554102cb0 2024-04-25T21:10:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.347+0000 7f754a7fc700 1 -- 172.21.15.47:0/732729261 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+942 (secure 0 0 0) 0x7f754409c210 con 0x7f7554102cb0 2024-04-25T21:10:40.347 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1752629947 2024-04-26T20:59:59.007729+0000 2024-04-25T21:10:40.347 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/831854467 2024-04-26T20:59:59.007729+0000 2024-04-25T21:10:40.347 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2517449296 2024-04-26T20:59:59.007729+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/253800519 2024-04-26T20:59:59.007729+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/253800519 2024-04-26T20:59:59.007729+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1508622913 2024-04-26T20:49:45.811145+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3318702222 2024-04-26T20:59:59.007729+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/2212759557 2024-04-26T20:49:45.811145+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/1508582803 2024-04-26T20:49:18.960239+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/1508582803 2024-04-26T20:49:18.960239+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1792012036 2024-04-26T20:49:18.960239+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2195749439 2024-04-26T20:49:45.811145+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2351249200 2024-04-26T20:49:45.811145+0000 2024-04-25T21:10:40.348 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2078121135 2024-04-26T20:49:18.960239+0000 2024-04-25T21:10:40.349 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/795796209 2024-04-26T20:49:18.960239+0000 2024-04-25T21:10:40.349 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/2212759557 2024-04-26T20:49:45.811145+0000 2024-04-25T21:10:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f75400a3e30 msgr2=0x7f75400a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f75400a3e30 0x7f75400a62f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7550005fd0 tx=0x7f7550005e30 comp rx=0 tx=0).stop 2024-04-25T21:10:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554102cb0 msgr2=0x7f75541997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554102cb0 0x7f75541997e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f754400d980 tx=0x7f754400dc90 comp rx=0 tx=0).stop 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 shutdown_connections 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f75400a3e30 0x7f75400a62f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7554106ec0 0x7f755419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7554102300 0x7f75541992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 --2- 172.21.15.47:0/732729261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7554102cb0 0x7f75541997e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 >> 172.21.15.47:0/732729261 conn(0x7f7554077f40 msgr2=0x7f75541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 shutdown_connections 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:40.351+0000 7f755b454700 1 -- 172.21.15.47:0/732729261 wait complete. 2024-04-25T21:10:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:listed 16 entries 2024-04-25T21:10:41.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:40 smithi092 ceph-mon[32524]: pgmap v331: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:41.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:40 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/732729261' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:10:41.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:40 smithi047 ceph-mon[29195]: pgmap v331: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:41.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:40 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/732729261' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:10:41.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:40 smithi186 ceph-mon[32529]: pgmap v331: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:41.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:40 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/732729261' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:10:41.275 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T21:10:41.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:10:41.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T21:10:41.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T21:10:41.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:10:41.355 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T21:10:41.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:10:41.397 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-25T21:10:41.466 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T21:10:41.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:10:41.467 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T21:10:41.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:10:41.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-25T21:10:41.643 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T21:10:41.643 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-25T21:10:41.643 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:10:41.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1226 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/b6bab0ba90685dc6fe0a022e536e6819b0df180b9ebd6343f32b00805aa5c4b5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/b6bab0ba90685dc6fe0a022e536e6819b0df180b9ebd6343f32b00805aa5c4b5/diff,workdir=/var/lib/containers/storage/overlay/b6bab0ba90685dc6fe0a022e536e6819b0df180b9ebd6343f32b00805aa5c4b5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/657b6f47eff61b1ee12fa3dfb241dc68c9bdc13ff6c310c4463cb628bdc6cbbd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/657b6f47eff61b1ee12fa3dfb241dc68c9bdc13ff6c310c4463cb628bdc6cbbd/diff,workdir=/var/lib/containers/storage/overlay/657b6f47eff61b1ee12fa3dfb241dc68c9bdc13ff6c310c4463cb628bdc6cbbd/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/ea334572038538fed6d3cf26cc6d27bc4e1f74db8c1c369c862c3feb060f6415/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/ea334572038538fed6d3cf26cc6d27bc4e1f74db8c1c369c862c3feb060f6415/diff,workdir=/var/lib/containers/storage/overlay/ea334572038538fed6d3cf26cc6d27bc4e1f74db8c1c369c862c3feb060f6415/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4397ce6475143e57199e4ac67946df16882dce30ecf6dfcdedac120d79d388d8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/4397ce6475143e57199e4ac67946df16882dce30ecf6dfcdedac120d79d388d8/diff,workdir=/var/lib/containers/storage/overlay/4397ce6475143e57199e4ac67946df16882dce30ecf6dfcdedac120d79d388d8/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/370c91bf77cc70f99aa75f4012f15f0bf419ac33144c46991750cd23527ee1e0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/370c91bf77cc70f99aa75f4012f15f0bf419ac33144c46991750cd23527ee1e0/diff,workdir=/var/lib/containers/storage/overlay/370c91bf77cc70f99aa75f4012f15f0bf419ac33144c46991750cd23527ee1e0/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d8d13238b8885ce94fb26817ca294e4dfa0ff13b1f76a3c0ab78ae9ebc748111/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/d8d13238b8885ce94fb26817ca294e4dfa0ff13b1f76a3c0ab78ae9ebc748111/diff,workdir=/var/lib/containers/storage/overlay/d8d13238b8885ce94fb26817ca294e4dfa0ff13b1f76a3c0ab78ae9ebc748111/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b95ccef49dab32793362ad06373049b4e4bad1183b3774ff86026a04d4ff130c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/b95ccef49dab32793362ad06373049b4e4bad1183b3774ff86026a04d4ff130c/diff,workdir=/var/lib/containers/storage/overlay/b95ccef49dab32793362ad06373049b4e4bad1183b3774ff86026a04d4ff130c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4e3b54cecae3edbfaaf7ac2de28883905a96106bd0ef2b834da8f7ac9890ddbe/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/4e3b54cecae3edbfaaf7ac2de28883905a96106bd0ef2b834da8f7ac9890ddbe/diff,workdir=/var/lib/containers/storage/overlay/4e3b54cecae3edbfaaf7ac2de28883905a96106bd0ef2b834da8f7ac9890ddbe/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/72c2438c932f8aedba4211b7fd7cd4711134a4ec4499465dcfb1b15e0cdb4f68/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/72c2438c932f8aedba4211b7fd7cd4711134a4ec4499465dcfb1b15e0cdb4f68/diff,workdir=/var/lib/containers/storage/overlay/72c2438c932f8aedba4211b7fd7cd4711134a4ec4499465dcfb1b15e0cdb4f68/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d50b37294fb37f45e1ba1cc25dc08c3262732336259fa72cff9541b9c227a570/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CKANTWA2BMN2R64S7POWHDTMQZ:/var/lib/containers/storage/overlay/l/M2OHWS5PJUQJH6LKP32SNOT6TQ,upperdir=/var/lib/containers/storage/overlay/d50b37294fb37f45e1ba1cc25dc08c3262732336259fa72cff9541b9c227a570/diff,workdir=/var/lib/containers/storage/overlay/d50b37294fb37f45e1ba1cc25dc08c3262732336259fa72cff9541b9c227a570/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-25T21:10:41.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:10:41.660 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-25T21:10:41.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:10:41.716 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-25T21:10:41.784 INFO:teuthology.orchestra.run.smithi047.stderr:Cannot find device "ceph-brx" 2024-04-25T21:10:41.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:10:41.785 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T21:10:41.796 INFO:tasks.cephadm:Teardown begin 2024-04-25T21:10:41.797 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T21:10:41.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:10:42.074 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:43.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:42 smithi092 ceph-mon[32524]: pgmap v332: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:43.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:42 smithi047 ceph-mon[29195]: pgmap v332: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:43.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:42 smithi186 ceph-mon[32529]: pgmap v332: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:10:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 -- 172.21.15.47:0/1888631502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3241034b0 msgr2=0x7fa32410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 --2- 172.21.15.47:0/1888631502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3241034b0 0x7fa32410f920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00b0d0 tx=0x7fa31c00b3e0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 -- 172.21.15.47:0/1888631502 shutdown_connections 2024-04-25T21:10:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 --2- 172.21.15.47:0/1888631502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3241034b0 0x7fa32410f920 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 --2- 172.21.15.47:0/1888631502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa324107940 0x7fa324107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 --2- 172.21.15.47:0/1888631502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa324102af0 0x7fa324102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 -- 172.21.15.47:0/1888631502 >> 172.21.15.47:0/1888631502 conn(0x7fa324079ba0 msgr2=0x7fa324079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 -- 172.21.15.47:0/1888631502 shutdown_connections 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.033+0000 7fa32c06b700 1 -- 172.21.15.47:0/1888631502 wait complete. 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32c06b700 1 Processor -- start 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32c06b700 1 -- start start 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32c06b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa324102af0 0x7fa32419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32c06b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3241034b0 0x7fa32419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32c06b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa324107940 0x7fa32419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32c06b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa324112c40 con 0x7fa324102af0 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32c06b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa324112ac0 con 0x7fa324107940 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32c06b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa324112dc0 con 0x7fa3241034b0 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329e07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa324102af0 0x7fa32419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329e07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa324102af0 0x7fa32419b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60132/0 (socket says 172.21.15.47:60132) 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329e07700 1 -- 172.21.15.47:0/664825677 learned_addr learned my addr 172.21.15.47:0/664825677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa32a608700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa324107940 0x7fa32419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329606700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3241034b0 0x7fa32419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329e07700 1 -- 172.21.15.47:0/664825677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3241034b0 msgr2=0x7fa32419b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329e07700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3241034b0 0x7fa32419b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329e07700 1 -- 172.21.15.47:0/664825677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa324107940 msgr2=0x7fa32419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329e07700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa324107940 0x7fa32419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.034+0000 7fa329e07700 1 -- 172.21.15.47:0/664825677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3241a0280 con 0x7fa324102af0 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.035+0000 7fa329e07700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa324102af0 0x7fa32419b2c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa32000d950 tx=0x7fa32000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.035+0000 7fa31affd700 1 -- 172.21.15.47:0/664825677 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320016070 con 0x7fa324102af0 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.035+0000 7fa31affd700 1 -- 172.21.15.47:0/664825677 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa32000fd40 con 0x7fa324102af0 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.035+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3241a0570 con 0x7fa324102af0 2024-04-25T21:10:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.035+0000 7fa31affd700 1 -- 172.21.15.47:0/664825677 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32000e3b0 con 0x7fa324102af0 2024-04-25T21:10:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.035+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241a7df0 con 0x7fa324102af0 2024-04-25T21:10:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.036+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa324069730 con 0x7fa324102af0 2024-04-25T21:10:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.037+0000 7fa31affd700 1 -- 172.21.15.47:0/664825677 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa320019070 con 0x7fa324102af0 2024-04-25T21:10:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.038+0000 7fa31affd700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa3100a3db0 0x7fa3100a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.038+0000 7fa329606700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa3100a3db0 0x7fa3100a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.038+0000 7fa31affd700 1 -- 172.21.15.47:0/664825677 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7fa3200d0230 con 0x7fa324102af0 2024-04-25T21:10:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.038+0000 7fa329606700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa3100a3db0 0x7fa3100a6270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa324076e10 tx=0x7fa31400a380 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.041+0000 7fa31affd700 1 -- 172.21.15.47:0/664825677 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa32009c8b0 con 0x7fa324102af0 2024-04-25T21:10:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.295+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3241a7fc0 con 0x7fa324102af0 2024-04-25T21:10:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.296+0000 7fa31affd700 1 -- 172.21.15.47:0/664825677 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11853 (secure 0 0 0) 0x7fa32009c1b0 con 0x7fa324102af0 2024-04-25T21:10:44.330 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:10:44.330 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T21:07:49.943471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa3100a3db0 msgr2=0x7fa3100a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa3100a3db0 0x7fa3100a6270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa324076e10 tx=0x7fa31400a380 comp rx=0 tx=0).stop 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa324102af0 msgr2=0x7fa32419b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa324102af0 0x7fa32419b2c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa32000d950 tx=0x7fa32000dc60 comp rx=0 tx=0).stop 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 shutdown_connections 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa3100a3db0 0x7fa3100a6270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa324102af0 0x7fa32419b2c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa324107940 0x7fa32419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 --2- 172.21.15.47:0/664825677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3241034b0 0x7fa32419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 >> 172.21.15.47:0/664825677 conn(0x7fa324079ba0 msgr2=0x7fa324100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 shutdown_connections 2024-04-25T21:10:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:44.298+0000 7fa32c06b700 1 -- 172.21.15.47:0/664825677 wait complete. 2024-04-25T21:10:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 38 2024-04-25T21:10:44.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:44 smithi047 ceph-mon[29195]: pgmap v333: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:44.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:44 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/664825677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:44.892 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:10:45.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:44 smithi092 ceph-mon[32524]: pgmap v333: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:45.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:44 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/664825677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:45.143 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:44 smithi186 ceph-mon[32529]: pgmap v333: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:10:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:44 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/664825677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:46.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 -- 172.21.15.47:0/3154717460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 msgr2=0x7fef1810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 --2- 172.21.15.47:0/3154717460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 0x7fef1810f8b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fef1000b0a0 tx=0x7fef1000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 -- 172.21.15.47:0/3154717460 shutdown_connections 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 --2- 172.21.15.47:0/3154717460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 0x7fef1810f8b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 --2- 172.21.15.47:0/3154717460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18108a50 0x7fef18108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 --2- 172.21.15.47:0/3154717460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef18102a90 0x7fef18102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 -- 172.21.15.47:0/3154717460 >> 172.21.15.47:0/3154717460 conn(0x7fef18079ba0 msgr2=0x7fef18079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 -- 172.21.15.47:0/3154717460 shutdown_connections 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.787+0000 7fef1efc1700 1 -- 172.21.15.47:0/3154717460 wait complete. 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1efc1700 1 Processor -- start 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1efc1700 1 -- start start 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1efc1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef18102a90 0x7fef18199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1efc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 0x7fef181996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1efc1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18108a50 0x7fef1819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1efc1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef18112be0 con 0x7fef18103450 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1efc1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef18112a60 con 0x7fef18102a90 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1efc1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fef18112d60 con 0x7fef18108a50 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef17fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 0x7fef181996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef17fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 0x7fef181996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60154/0 (socket says 172.21.15.47:60154) 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef17fff700 1 -- 172.21.15.47:0/1091693911 learned_addr learned my addr 172.21.15.47:0/1091693911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1d55e700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18108a50 0x7fef1819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1cd5d700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef18102a90 0x7fef18199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef17fff700 1 -- 172.21.15.47:0/1091693911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18108a50 msgr2=0x7fef1819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef17fff700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18108a50 0x7fef1819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef17fff700 1 -- 172.21.15.47:0/1091693911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef18102a90 msgr2=0x7fef18199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef17fff700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef18102a90 0x7fef18199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef17fff700 1 -- 172.21.15.47:0/1091693911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1819e180 con 0x7fef18103450 2024-04-25T21:10:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.788+0000 7fef1d55e700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18108a50 0x7fef1819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.789+0000 7fef17fff700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 0x7fef181996b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fef0400d950 tx=0x7fef0400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.789+0000 7fef15ffb700 1 -- 172.21.15.47:0/1091693911 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04016070 con 0x7fef18103450 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.789+0000 7fef15ffb700 1 -- 172.21.15.47:0/1091693911 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef04009df0 con 0x7fef18103450 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.789+0000 7fef15ffb700 1 -- 172.21.15.47:0/1091693911 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0400f3c0 con 0x7fef18103450 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.789+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1819e470 con 0x7fef18103450 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.789+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef181a5cf0 con 0x7fef18103450 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.790+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef18069730 con 0x7fef18103450 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.791+0000 7fef15ffb700 1 -- 172.21.15.47:0/1091693911 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fef04019070 con 0x7fef18103450 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.792+0000 7fef15ffb700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fef080a3e30 0x7fef080a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.792+0000 7fef1cd5d700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fef080a3e30 0x7fef080a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.792+0000 7fef15ffb700 1 -- 172.21.15.47:0/1091693911 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7fef040d0380 con 0x7fef18103450 2024-04-25T21:10:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.792+0000 7fef1cd5d700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fef080a3e30 0x7fef080a62f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005950 tx=0x7fef0c00a300 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:46.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:46.796+0000 7fef15ffb700 1 -- 172.21.15.47:0/1091693911 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef0409ca00 con 0x7fef18103450 2024-04-25T21:10:46.834 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:46 smithi047 ceph-mon[29195]: pgmap v334: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:10:47.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:46 smithi092 ceph-mon[32524]: pgmap v334: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:10:47.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.052+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef1819a4d0 con 0x7fef18103450 2024-04-25T21:10:47.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.053+0000 7fef15ffb700 1 -- 172.21.15.47:0/1091693911 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11853 (secure 0 0 0) 0x7fef0409c300 con 0x7fef18103450 2024-04-25T21:10:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:10:47.053 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/911765730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":911765730},{"type":"v1","addr":"172.21.15.47:6835","nonce":911765730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/12948729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":12948729},{"type":"v1","addr":"172.21.15.47:6837","nonce":12948729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2858318196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2858318196},{"type":"v1","addr":"172.21.15.47:6839","nonce":2858318196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/622157648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":622157648},{"type":"v1","addr":"172.21.15.47:6841","nonce":622157648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3007060622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3007060622},{"type":"v1","addr":"172.21.15.186:6833","nonce":3007060622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1380167956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1380167956},{"type":"v1","addr":"172.21.15.92:6833","nonce":1380167956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:56:53.085968+0000","modified":"2024-04-25T21:07:49.943471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24467,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3860377657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3860377657},{"type":"v1","addr":"172.21.15.186:6835","nonce":3860377657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/4095064414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4095064414},{"type":"v1","addr":"172.21.15.186:6837","nonce":4095064414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/60292465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":60292465},{"type":"v1","addr":"172.21.15.186:6839","nonce":60292465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.92:6835/2097108178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2097108178},{"type":"v1","addr":"172.21.15.92:6835","nonce":2097108178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1984144918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1984144918},{"type":"v1","addr":"172.21.15.92:6837","nonce":1984144918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/941998037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":941998037},{"type":"v1","addr":"172.21.15.92:6839","nonce":941998037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24452,24467,24488]},"id":1}]} 2024-04-25T21:10:47.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fef080a3e30 msgr2=0x7fef080a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:47.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fef080a3e30 0x7fef080a62f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005950 tx=0x7fef0c00a300 comp rx=0 tx=0).stop 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 msgr2=0x7fef181996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 0x7fef181996b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fef0400d950 tx=0x7fef0400dc60 comp rx=0 tx=0).stop 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 shutdown_connections 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fef080a3e30 0x7fef080a62f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18108a50 0x7fef1819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef18102a90 0x7fef18199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 --2- 172.21.15.47:0/1091693911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef18103450 0x7fef181996b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 >> 172.21.15.47:0/1091693911 conn(0x7fef18079ba0 msgr2=0x7fef181018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 shutdown_connections 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:47.055+0000 7fef1efc1700 1 -- 172.21.15.47:0/1091693911 wait complete. 2024-04-25T21:10:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 38 2024-04-25T21:10:47.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:46 smithi186 ceph-mon[32529]: pgmap v334: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:10:47.737 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:10:47.995 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:48.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:47 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1091693911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:48.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:47 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1091693911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:48.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:47 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1091693911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:49.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:48 smithi092 ceph-mon[32524]: pgmap v335: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:10:49.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:48 smithi047 ceph-mon[29195]: pgmap v335: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:10:49.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:48 smithi186 ceph-mon[32529]: pgmap v335: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:10:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 -- 172.21.15.47:0/3450409505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 msgr2=0x7fb5ec108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3450409505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 0x7fb5ec108e40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009990 tx=0x7fb5dc009ca0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 -- 172.21.15.47:0/3450409505 shutdown_connections 2024-04-25T21:10:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3450409505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 0x7fb5ec108e40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3450409505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec06bbb0 0x7fb5ec10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3450409505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5ec06b1f0 0x7fb5ec06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 -- 172.21.15.47:0/3450409505 >> 172.21.15.47:0/3450409505 conn(0x7fb5ec0789e0 msgr2=0x7fb5ec078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 -- 172.21.15.47:0/3450409505 shutdown_connections 2024-04-25T21:10:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.460+0000 7fb5f1137700 1 -- 172.21.15.47:0/3450409505 wait complete. 2024-04-25T21:10:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.461+0000 7fb5f1137700 1 Processor -- start 2024-04-25T21:10:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.461+0000 7fb5f1137700 1 -- start start 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5f1137700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec06b1f0 0x7fb5ec1019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5f1137700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5ec06bbb0 0x7fb5ec101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5f1137700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 0x7fb5ec102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5f1137700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec112ca0 con 0x7fb5ec108a60 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5f1137700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec112b20 con 0x7fb5ec06b1f0 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5f1137700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec112e20 con 0x7fb5ec06bbb0 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 0x7fb5ec102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 0x7fb5ec102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60168/0 (socket says 172.21.15.47:60168) 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 -- 172.21.15.47:0/3046574000 learned_addr learned my addr 172.21.15.47:0/3046574000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5ea59c700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5ec06bbb0 0x7fb5ec101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5ead9d700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec06b1f0 0x7fb5ec1019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 -- 172.21.15.47:0/3046574000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5ec06bbb0 msgr2=0x7fb5ec101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5ec06bbb0 0x7fb5ec101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 -- 172.21.15.47:0/3046574000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec06b1f0 msgr2=0x7fb5ec1019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec06b1f0 0x7fb5ec1019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 -- 172.21.15.47:0/3046574000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec1a7710 con 0x7fb5ec108a60 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.462+0000 7fb5eb59e700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 0x7fb5ec102430 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4007ee0 tx=0x7fb5e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.463+0000 7fb5d3fff700 1 -- 172.21.15.47:0/3046574000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400dbd0 con 0x7fb5ec108a60 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.463+0000 7fb5d3fff700 1 -- 172.21.15.47:0/3046574000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e400dd70 con 0x7fb5ec108a60 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.463+0000 7fb5d3fff700 1 -- 172.21.15.47:0/3046574000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4005990 con 0x7fb5ec108a60 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.463+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec1a7a00 con 0x7fb5ec108a60 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.463+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1a7ef0 con 0x7fb5ec108a60 2024-04-25T21:10:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.464+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ec10b060 con 0x7fb5ec108a60 2024-04-25T21:10:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.465+0000 7fb5d3fff700 1 -- 172.21.15.47:0/3046574000 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb5e401a020 con 0x7fb5ec108a60 2024-04-25T21:10:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.465+0000 7fb5d3fff700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb5d40a3e30 0x7fb5d40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.466+0000 7fb5ead9d700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb5d40a3e30 0x7fb5d40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.466+0000 7fb5d3fff700 1 -- 172.21.15.47:0/3046574000 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7fb5e40d0680 con 0x7fb5ec108a60 2024-04-25T21:10:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.466+0000 7fb5ead9d700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb5d40a3e30 0x7fb5d40a62f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc006010 tx=0x7fb5dc01a040 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.469+0000 7fb5d3fff700 1 -- 172.21.15.47:0/3046574000 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5e4010070 con 0x7fb5ec108a60 2024-04-25T21:10:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.680+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5ec02df80 con 0x7fb5ec108a60 2024-04-25T21:10:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.682+0000 7fb5d3fff700 1 -- 172.21.15.47:0/3046574000 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23065 (secure 0 0 0) 0x7fb5e409cca0 con 0x7fb5ec108a60 2024-04-25T21:10:49.682 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-25T21:10:49.682 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":100,"fsid":"0ca8de06-0345-11ef-bc93-c7b262605968","created":"2024-04-25T20:48:51.766698+0000","modified":"2024-04-25T21:10:11.844213+0000","last_up_change":"2024-04-25T20:55:48.271140+0000","last_in_change":"2024-04-25T20:55:28.692481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:52:09.901826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:56:46.327630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:56:49.911562+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4b824d92-28a7-4a8e-b2fe-1d6512481dea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6803","nonce":3737942264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6805","nonce":3737942264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6809","nonce":3737942264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3737942264},{"type":"v1","addr":"172.21.15.47:6807","nonce":3737942264}]},"public_addr":"172.21.15.47:6803/3737942264","cluster_addr":"172.21.15.47:6805/3737942264","heartbeat_back_addr":"172.21.15.47:6809/3737942264","heartbeat_front_addr":"172.21.15.47:6807/3737942264","state":["exists","up"]},{"osd":1,"uuid":"cef4a330-707b-41e0-bf53-12a1a03da237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6801","nonce":3070209756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6803","nonce":3070209756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6807","nonce":3070209756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3070209756},{"type":"v1","addr":"172.21.15.92:6805","nonce":3070209756}]},"public_addr":"172.21.15.92:6801/3070209756","cluster_addr":"172.21.15.92:6803/3070209756","heartbeat_back_addr":"172.21.15.92:6807/3070209756","heartbeat_front_addr":"172.21.15.92:6805/3070209756","state":["exists","up"]},{"osd":2,"uuid":"49a4bac4-8c22-47d9-b96f-56e99bd8cec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6801","nonce":651738121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6803","nonce":651738121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6807","nonce":651738121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":651738121},{"type":"v1","addr":"172.21.15.186:6805","nonce":651738121}]},"public_addr":"172.21.15.186:6801/651738121","cluster_addr":"172.21.15.186:6803/651738121","heartbeat_back_addr":"172.21.15.186:6807/651738121","heartbeat_front_addr":"172.21.15.186:6805/651738121","state":["exists","up"]},{"osd":3,"uuid":"0f303ea7-0cef-436c-82ed-e3a7f1fde8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6811","nonce":2086583036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6813","nonce":2086583036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6817","nonce":2086583036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2086583036},{"type":"v1","addr":"172.21.15.47:6815","nonce":2086583036}]},"public_addr":"172.21.15.47:6811/2086583036","cluster_addr":"172.21.15.47:6813/2086583036","heartbeat_back_addr":"172.21.15.47:6817/2086583036","heartbeat_front_addr":"172.21.15.47:6815/2086583036","state":["exists","up"]},{"osd":4,"uuid":"8caf5e2e-b43b-4b0e-8ade-0ce8e6988c05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6809","nonce":477256695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6811","nonce":477256695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6815","nonce":477256695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":477256695},{"type":"v1","addr":"172.21.15.92:6813","nonce":477256695}]},"public_addr":"172.21.15.92:6809/477256695","cluster_addr":"172.21.15.92:6811/477256695","heartbeat_back_addr":"172.21.15.92:6815/477256695","heartbeat_front_addr":"172.21.15.92:6813/477256695","state":["exists","up"]},{"osd":5,"uuid":"21521287-3d9f-4f8d-aa39-4a2d79d2cf34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6809","nonce":2592795959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6811","nonce":2592795959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6815","nonce":2592795959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2592795959},{"type":"v1","addr":"172.21.15.186:6813","nonce":2592795959}]},"public_addr":"172.21.15.186:6809/2592795959","cluster_addr":"172.21.15.186:6811/2592795959","heartbeat_back_addr":"172.21.15.186:6815/2592795959","heartbeat_front_addr":"172.21.15.186:6813/2592795959","state":["exists","up"]},{"osd":6,"uuid":"a7726429-12e9-4dd8-9bae-8aa979f0c447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6819","nonce":2069873922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6821","nonce":2069873922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6825","nonce":2069873922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2069873922},{"type":"v1","addr":"172.21.15.47:6823","nonce":2069873922}]},"public_addr":"172.21.15.47:6819/2069873922","cluster_addr":"172.21.15.47:6821/2069873922","heartbeat_back_addr":"172.21.15.47:6825/2069873922","heartbeat_front_addr":"172.21.15.47:6823/2069873922","state":["exists","up"]},{"osd":7,"uuid":"e9f86109-f4e5-4770-88b9-6454fa1927bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6817","nonce":1310010410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6819","nonce":1310010410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6823","nonce":1310010410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1310010410},{"type":"v1","addr":"172.21.15.92:6821","nonce":1310010410}]},"public_addr":"172.21.15.92:6817/1310010410","cluster_addr":"172.21.15.92:6819/1310010410","heartbeat_back_addr":"172.21.15.92:6823/1310010410","heartbeat_front_addr":"172.21.15.92:6821/1310010410","state":["exists","up"]},{"osd":8,"uuid":"df80bc74-f3eb-43e1-ac44-7061cdddf622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6817","nonce":773299526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6819","nonce":773299526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6823","nonce":773299526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":773299526},{"type":"v1","addr":"172.21.15.186:6821","nonce":773299526}]},"public_addr":"172.21.15.186:6817/773299526","cluster_addr":"172.21.15.186:6819/773299526","heartbeat_back_addr":"172.21.15.186:6823/773299526","heartbeat_front_addr":"172.21.15.186:6821/773299526","state":["exists","up"]},{"osd":9,"uuid":"b313eacf-dc05-4b28-a5d4-4d04e7b1bda9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6827","nonce":3118226797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6829","nonce":3118226797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6833","nonce":3118226797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3118226797},{"type":"v1","addr":"172.21.15.47:6831","nonce":3118226797}]},"public_addr":"172.21.15.47:6827/3118226797","cluster_addr":"172.21.15.47:6829/3118226797","heartbeat_back_addr":"172.21.15.47:6833/3118226797","heartbeat_front_addr":"172.21.15.47:6831/3118226797","state":["exists","up"]},{"osd":10,"uuid":"7a31b12e-d889-4a51-9060-b79255c81626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6825","nonce":3185760943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6827","nonce":3185760943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6831","nonce":3185760943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3185760943},{"type":"v1","addr":"172.21.15.92:6829","nonce":3185760943}]},"public_addr":"172.21.15.92:6825/3185760943","cluster_addr":"172.21.15.92:6827/3185760943","heartbeat_back_addr":"172.21.15.92:6831/3185760943","heartbeat_front_addr":"172.21.15.92:6829/3185760943","state":["exists","up"]},{"osd":11,"uuid":"92dc80e5-665e-412c-9596-b4ea0f65e561","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6825","nonce":936644453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6827","nonce":936644453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6831","nonce":936644453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":936644453},{"type":"v1","addr":"172.21.15.186:6829","nonce":936644453}]},"public_addr":"172.21.15.186:6825/936644453","cluster_addr":"172.21.15.186:6827/936644453","heartbeat_back_addr":"172.21.15.186:6831/936644453","heartbeat_front_addr":"172.21.15.186:6829/936644453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:19.762831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:43.654422+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:07.376570+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:31.216695+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:55.341348+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:19.517142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:43.192125+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:07.574937+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:31.914700+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:54:56.349337+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:21.638748+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:55:46.798930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1752629947":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/831854467":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/2517449296":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6801/253800519":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6800/253800519":"2024-04-26T20:59:59.007729+0000","172.21.15.47:0/1508622913":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/3318702222":"2024-04-26T20:59:59.007729+0000","172.21.15.47:6801/2212759557":"2024-04-26T20:49:45.811145+0000","172.21.15.47:6800/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6801/1508582803":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/1792012036":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/2195749439":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2351249200":"2024-04-26T20:49:45.811145+0000","172.21.15.47:0/2078121135":"2024-04-26T20:49:18.960239+0000","172.21.15.47:0/795796209":"2024-04-26T20:49:18.960239+0000","172.21.15.47:6800/2212759557":"2024-04-26T20:49:45.811145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:10:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb5d40a3e30 msgr2=0x7fb5d40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb5d40a3e30 0x7fb5d40a62f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc006010 tx=0x7fb5dc01a040 comp rx=0 tx=0).stop 2024-04-25T21:10:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 msgr2=0x7fb5ec102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 0x7fb5ec102430 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4007ee0 tx=0x7fb5e400d3b0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 shutdown_connections 2024-04-25T21:10:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fb5d40a3e30 0x7fb5d40a62f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5ec108a60 0x7fb5ec102430 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec06b1f0 0x7fb5ec1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 --2- 172.21.15.47:0/3046574000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5ec06bbb0 0x7fb5ec101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 >> 172.21.15.47:0/3046574000 conn(0x7fb5ec0789e0 msgr2=0x7fb5ec0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 shutdown_connections 2024-04-25T21:10:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:49.684+0000 7fb5f1137700 1 -- 172.21.15.47:0/3046574000 wait complete. 2024-04-25T21:10:49.996 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:49 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3046574000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:10:50.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:49 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3046574000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:10:50.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:49 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3046574000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:10:50.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-25T21:10:50.569 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:51.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:50 smithi047 ceph-mon[29195]: pgmap v336: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:10:51.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:50 smithi186 ceph-mon[32529]: pgmap v336: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:10:51.281 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:50 smithi092 ceph-mon[32524]: pgmap v336: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:10:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 -- 172.21.15.47:0/1189055882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8107c10 msgr2=0x7fdfe8107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 --2- 172.21.15.47:0/1189055882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8107c10 0x7fdfe8107ff0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400b0d0 tx=0x7fdfe400b3e0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 -- 172.21.15.47:0/1189055882 shutdown_connections 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 --2- 172.21.15.47:0/1189055882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8107c10 0x7fdfe8107ff0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 --2- 172.21.15.47:0/1189055882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe8102eb0 0x7fdfe810e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 --2- 172.21.15.47:0/1189055882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfe81024f0 0x7fdfe8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 -- 172.21.15.47:0/1189055882 >> 172.21.15.47:0/1189055882 conn(0x7fdfe8076c60 msgr2=0x7fdfe8077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 -- 172.21.15.47:0/1189055882 shutdown_connections 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.119+0000 7fdff03ba700 1 -- 172.21.15.47:0/1189055882 wait complete. 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdff03ba700 1 Processor -- start 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdff03ba700 1 -- start start 2024-04-25T21:10:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdff03ba700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfe81024f0 0x7fdfe8103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdff03ba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8102eb0 0x7fdfe8103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdff03ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe8107c10 0x7fdfe81040f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdff03ba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8111ca0 con 0x7fdfe8102eb0 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdff03ba700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8111b20 con 0x7fdfe8107c10 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdff03ba700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8111e20 con 0x7fdfe81024f0 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdfed955700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8102eb0 0x7fdfe8103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdfed955700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8102eb0 0x7fdfe8103bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60186/0 (socket says 172.21.15.47:60186) 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.120+0000 7fdfed955700 1 -- 172.21.15.47:0/7817326 learned_addr learned my addr 172.21.15.47:0/7817326 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfee156700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfe81024f0 0x7fdfe8103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfee957700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe8107c10 0x7fdfe81040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfed955700 1 -- 172.21.15.47:0/7817326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfe81024f0 msgr2=0x7fdfe8103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfed955700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfe81024f0 0x7fdfe8103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfed955700 1 -- 172.21.15.47:0/7817326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe8107c10 msgr2=0x7fdfe81040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfed955700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe8107c10 0x7fdfe81040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfed955700 1 -- 172.21.15.47:0/7817326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81a7750 con 0x7fdfe8102eb0 2024-04-25T21:10:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfed955700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8102eb0 0x7fdfe8103bb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800d980 tx=0x7fdfd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfdf7fe700 1 -- 172.21.15.47:0/7817326 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8017070 con 0x7fdfe8102eb0 2024-04-25T21:10:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe81a7a40 con 0x7fdfe8102eb0 2024-04-25T21:10:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfdf7fe700 1 -- 172.21.15.47:0/7817326 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd8005020 con 0x7fdfe8102eb0 2024-04-25T21:10:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdfdf7fe700 1 -- 172.21.15.47:0/7817326 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8009b50 con 0x7fdfe8102eb0 2024-04-25T21:10:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.121+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe81a7f30 con 0x7fdfe8102eb0 2024-04-25T21:10:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.122+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe8069730 con 0x7fdfe8102eb0 2024-04-25T21:10:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.123+0000 7fdfdf7fe700 1 -- 172.21.15.47:0/7817326 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdfd80051d0 con 0x7fdfe8102eb0 2024-04-25T21:10:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.124+0000 7fdfdf7fe700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdfd40a3db0 0x7fdfd40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.124+0000 7fdfee156700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdfd40a3db0 0x7fdfd40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.124+0000 7fdfdf7fe700 1 -- 172.21.15.47:0/7817326 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7fdfd80d0ae0 con 0x7fdfe8102eb0 2024-04-25T21:10:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.125+0000 7fdfee156700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdfd40a3db0 0x7fdfd40a6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009fd0 tx=0x7fdfe400b040 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.128+0000 7fdfdf7fe700 1 -- 172.21.15.47:0/7817326 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfd8020020 con 0x7fdfe8102eb0 2024-04-25T21:10:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.383+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdfe802df80 con 0x7fdfe8102eb0 2024-04-25T21:10:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.385+0000 7fdfdf7fe700 1 -- 172.21.15.47:0/7817326 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7fdfd809d160 con 0x7fdfe8102eb0 2024-04-25T21:10:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdfd40a3db0 msgr2=0x7fdfd40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdfd40a3db0 0x7fdfd40a6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009fd0 tx=0x7fdfe400b040 comp rx=0 tx=0).stop 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8102eb0 msgr2=0x7fdfe8103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8102eb0 0x7fdfe8103bb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800d980 tx=0x7fdfd800dc90 comp rx=0 tx=0).stop 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 shutdown_connections 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fdfd40a3db0 0x7fdfd40a6270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe8107c10 0x7fdfe81040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfe81024f0 0x7fdfe8103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 --2- 172.21.15.47:0/7817326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8102eb0 0x7fdfe8103bb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 >> 172.21.15.47:0/7817326 conn(0x7fdfe8076c60 msgr2=0x7fdfe8101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 shutdown_connections 2024-04-25T21:10:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:52.388+0000 7fdff03ba700 1 -- 172.21.15.47:0/7817326 wait complete. 2024-04-25T21:10:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T21:10:52.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T21:10:52.383+0000 7f8675144700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:10:52.719 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c[62450]: ignoring --setuser ceph since I am not root 2024-04-25T21:10:52.720 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c[62450]: ignoring --setgroup ceph since I am not root 2024-04-25T21:10:52.720 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mds[62454]: -- 172.21.15.186:0/4207173587 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555db4253a40 con 0x555db4227000 2024-04-25T21:10:52.720 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c[62450]: starting mds.c at 2024-04-25T21:10:52.720 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f[62141]: ignoring --setuser ceph since I am not root 2024-04-25T21:10:52.720 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f[62141]: ignoring --setgroup ceph since I am not root 2024-04-25T21:10:52.720 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mds[62145]: -- 172.21.15.186:0/43051541 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608bf09da40 con 0x5608bf070800 2024-04-25T21:10:52.720 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f[62141]: starting mds.f at 2024-04-25T21:10:52.720 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i[62769]: ignoring --setuser ceph since I am not root 2024-04-25T21:10:52.721 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i[62769]: ignoring --setgroup ceph since I am not root 2024-04-25T21:10:52.721 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mds[62773]: -- 172.21.15.186:0/2781144343 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558fa9885a40 con 0x558fa9858c00 2024-04-25T21:10:52.721 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i[62769]: starting mds.i at 2024-04-25T21:10:52.721 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i[62769]: 2024-04-25T21:10:52.436+0000 7f3f3ee4d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-04-25T21:10:52.782 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-e[62181]: ignoring --setuser ceph since I am not root 2024-04-25T21:10:52.782 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-e[62181]: ignoring --setgroup ceph since I am not root 2024-04-25T21:10:52.782 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mds[62185]: -- 172.21.15.92:0/1996388396 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55684ea91a40 con 0x55684ea64800 2024-04-25T21:10:52.782 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-e[62181]: starting mds.e at 2024-04-25T21:10:52.782 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-h[62485]: ignoring --setuser ceph since I am not root 2024-04-25T21:10:52.782 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-h[62485]: ignoring --setgroup ceph since I am not root 2024-04-25T21:10:52.782 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mds[62489]: -- 172.21.15.92:0/2362813558 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d336355a40 con 0x55d336328800 2024-04-25T21:10:52.782 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-h[62485]: starting mds.h at 2024-04-25T21:10:53.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:52 smithi047 ceph-mon[29195]: pgmap v337: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:10:53.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:52 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/7817326' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:10:53.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:52 smithi047 ceph-mon[29195]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:10:53.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:52 smithi047 ceph-mon[29195]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:10:53.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:52 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/7817326' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:10:53.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:52 smithi047 ceph-mon[29195]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:10:53.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:52 smithi047 ceph-mon[29195]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T21:10:53.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T21:10:53.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mon[32529]: pgmap v337: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:10:53.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/7817326' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:10:53.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mon[32529]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:10:53.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mon[32529]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:10:53.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/7817326' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:10:53.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mon[32529]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:10:53.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:52 smithi186 ceph-mon[32529]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T21:10:53.281 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mon[32524]: pgmap v337: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:10:53.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/7817326' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:10:53.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mon[32524]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:10:53.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mon[32524]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:10:53.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/7817326' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:10:53.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mon[32524]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:10:53.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:52 smithi092 ceph-mon[32524]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T21:10:53.284 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:54.579 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: pgmap v339: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-25T21:10:54.579 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.186:6834/1617787605,v1:172.21.15.186:6837/1617787605] up:boot 2024-04-25T21:10:54.579 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.186:6836/3881266986,v1:172.21.15.186:6839/3881266986] up:boot 2024-04-25T21:10:54.579 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.186:6835/3154122388,v1:172.21.15.186:6838/3154122388] up:boot 2024-04-25T21:10:54.579 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6834/2290793347,v1:172.21.15.92:6835/2290793347] up:boot 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6836/1481114484,v1:172.21.15.92:6837/1481114484] up:boot 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: mds.? [v2:172.21.15.92:6838/3548445396,v1:172.21.15.92:6839/3548445396] up:boot 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:10:54.580 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:54 smithi047 ceph-mon[29195]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: pgmap v339: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.186:6834/1617787605,v1:172.21.15.186:6837/1617787605] up:boot 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.186:6836/3881266986,v1:172.21.15.186:6839/3881266986] up:boot 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.186:6835/3154122388,v1:172.21.15.186:6838/3154122388] up:boot 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6834/2290793347,v1:172.21.15.92:6835/2290793347] up:boot 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6836/1481114484,v1:172.21.15.92:6837/1481114484] up:boot 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: mds.? [v2:172.21.15.92:6838/3548445396,v1:172.21.15.92:6839/3548445396] up:boot 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:10:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:10:54.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:10:54.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:10:54.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:54 smithi186 ceph-mon[32529]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:10:54.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: pgmap v339: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.186:6834/1617787605,v1:172.21.15.186:6837/1617787605] up:boot 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.186:6836/3881266986,v1:172.21.15.186:6839/3881266986] up:boot 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.186:6835/3154122388,v1:172.21.15.186:6838/3154122388] up:boot 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6834/2290793347,v1:172.21.15.92:6835/2290793347] up:boot 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6836/1481114484,v1:172.21.15.92:6837/1481114484] up:boot 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: mds.? [v2:172.21.15.92:6838/3548445396,v1:172.21.15.92:6839/3548445396] up:boot 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:10:54.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:54 smithi092 ceph-mon[32524]: from='mgr.24811 172.21.15.47:0/2672226266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:10:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3478840648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 msgr2=0x7fd188108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3478840648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 0x7fd188108e50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd184009990 tx=0x7fd184009ca0 comp rx=0 tx=0).stop 2024-04-25T21:10:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3478840648 shutdown_connections 2024-04-25T21:10:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3478840648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 0x7fd188108e50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3478840648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18806bb30 0x7fd18810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3478840648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd18806b170 0x7fd18806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3478840648 >> 172.21.15.47:0/3478840648 conn(0x7fd1880777b0 msgr2=0x7fd188077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3478840648 shutdown_connections 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.819+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3478840648 wait complete. 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18fb1c700 1 Processor -- start 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18fb1c700 1 -- start start 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18fb1c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18806b170 0x7fd188101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18fb1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd18806bb30 0x7fd1881022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18fb1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 0x7fd1881027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18fb1c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd188112c80 con 0x7fd188108a70 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18fb1c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd188112b00 con 0x7fd18806bb30 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18fb1c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd188112e00 con 0x7fd18806b170 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18e0b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 0x7fd1881027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18e0b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 0x7fd1881027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36496/0 (socket says 172.21.15.47:36496) 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18e0b9700 1 -- 172.21.15.47:0/3071765364 learned_addr learned my addr 172.21.15.47:0/3071765364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18d0b7700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd18806bb30 0x7fd1881022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18d8b8700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18806b170 0x7fd188101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18e0b9700 1 -- 172.21.15.47:0/3071765364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18806b170 msgr2=0x7fd188101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.820+0000 7fd18e0b9700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18806b170 0x7fd188101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd18e0b9700 1 -- 172.21.15.47:0/3071765364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd18806bb30 msgr2=0x7fd1881022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd18e0b9700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd18806bb30 0x7fd1881022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd18e0b9700 1 -- 172.21.15.47:0/3071765364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1881a76c0 con 0x7fd188108a70 2024-04-25T21:10:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd18e0b9700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 0x7fd1881027e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd180007ee0 tx=0x7fd18000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd17effd700 1 -- 172.21.15.47:0/3071765364 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180004bb0 con 0x7fd188108a70 2024-04-25T21:10:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd17effd700 1 -- 172.21.15.47:0/3071765364 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd180004d50 con 0x7fd188108a70 2024-04-25T21:10:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1881a79b0 con 0x7fd188108a70 2024-04-25T21:10:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd17effd700 1 -- 172.21.15.47:0/3071765364 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180005920 con 0x7fd188108a70 2024-04-25T21:10:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.821+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1881a7ef0 con 0x7fd188108a70 2024-04-25T21:10:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.822+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1881041a0 con 0x7fd188108a70 2024-04-25T21:10:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.823+0000 7fd17effd700 1 -- 172.21.15.47:0/3071765364 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd18001a020 con 0x7fd188108a70 2024-04-25T21:10:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.824+0000 7fd17effd700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fd1740a3e30 0x7fd1740a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.824+0000 7fd18d8b8700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fd1740a3e30 0x7fd1740a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.824+0000 7fd17effd700 1 -- 172.21.15.47:0/3071765364 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8764+0+0 (secure 0 0 0) 0x7fd1800d05a0 con 0x7fd188108a70 2024-04-25T21:10:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.824+0000 7fd18d8b8700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fd1740a3e30 0x7fd1740a62f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b5c0 tx=0x7fd184005fd0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:54.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:54.827+0000 7fd17effd700 1 -- 172.21.15.47:0/3071765364 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd180010070 con 0x7fd188108a70 2024-04-25T21:10:55.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.086+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd18802df80 con 0x7fd188108a70 2024-04-25T21:10:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.394+0000 7fd17effd700 1 -- 172.21.15.47:0/3071765364 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7fd18009caf0 con 0x7fd188108a70 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fd1740a3e30 msgr2=0x7fd1740a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fd1740a3e30 0x7fd1740a62f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b5c0 tx=0x7fd184005fd0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 msgr2=0x7fd1881027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 0x7fd1881027e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd180007ee0 tx=0x7fd18000d3b0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 shutdown_connections 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fd1740a3e30 0x7fd1740a62f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd188108a70 0x7fd1881027e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18806b170 0x7fd188101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 --2- 172.21.15.47:0/3071765364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd18806bb30 0x7fd1881022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.396+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 >> 172.21.15.47:0/3071765364 conn(0x7fd1880777b0 msgr2=0x7fd188100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.397+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 shutdown_connections 2024-04-25T21:10:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:55.397+0000 7fd18fb1c700 1 -- 172.21.15.47:0/3071765364 wait complete. 2024-04-25T21:10:55.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:55 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3071765364' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:10:55.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:55 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3071765364' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:10:55.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:55 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3071765364' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:10:56.026 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T21:10:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:56.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:56 smithi047 ceph-mon[29195]: pgmap v340: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T21:10:56.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:56 smithi047 ceph-mon[29195]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:10:56.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:56 smithi047 ceph-mon[29195]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:10:56.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:56 smithi047 ceph-mon[29195]: Cluster is now healthy 2024-04-25T21:10:56.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:56 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/3071765364' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:10:56.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:56 smithi047 ceph-mon[29195]: fsmap 12 up:standby 2024-04-25T21:10:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:56 smithi186 ceph-mon[32529]: pgmap v340: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T21:10:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:56 smithi186 ceph-mon[32529]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:10:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:56 smithi186 ceph-mon[32529]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:10:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:56 smithi186 ceph-mon[32529]: Cluster is now healthy 2024-04-25T21:10:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:56 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/3071765364' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:10:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:56 smithi186 ceph-mon[32529]: fsmap 12 up:standby 2024-04-25T21:10:56.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:56 smithi092 ceph-mon[32524]: pgmap v340: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T21:10:56.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:56 smithi092 ceph-mon[32524]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:10:56.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:56 smithi092 ceph-mon[32524]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:10:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:56 smithi092 ceph-mon[32524]: Cluster is now healthy 2024-04-25T21:10:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:56 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/3071765364' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:10:56.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:56 smithi092 ceph-mon[32524]: fsmap 12 up:standby 2024-04-25T21:10:57.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 -- 172.21.15.47:0/1989856290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064108a70 msgr2=0x7fa064108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:57.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1989856290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064108a70 0x7fa064108e50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa060009990 tx=0x7fa060009ca0 comp rx=0 tx=0).stop 2024-04-25T21:10:57.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 -- 172.21.15.47:0/1989856290 shutdown_connections 2024-04-25T21:10:57.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1989856290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064108a70 0x7fa064108e50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1989856290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa064103480 0x7fa06410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1989856290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa064102ac0 0x7fa064102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 -- 172.21.15.47:0/1989856290 >> 172.21.15.47:0/1989856290 conn(0x7fa064079bb0 msgr2=0x7fa064079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 -- 172.21.15.47:0/1989856290 shutdown_connections 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.853+0000 7fa06baa6700 1 -- 172.21.15.47:0/1989856290 wait complete. 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06baa6700 1 Processor -- start 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06baa6700 1 -- start start 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06baa6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa064102ac0 0x7fa064199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06baa6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064103480 0x7fa0641995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06baa6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa064108a70 0x7fa06419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06baa6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa064112c00 con 0x7fa064103480 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06baa6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa064112a80 con 0x7fa064102ac0 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06baa6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa064112d80 con 0x7fa064108a70 2024-04-25T21:10:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069041700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064103480 0x7fa0641995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069041700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064103480 0x7fa0641995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36522/0 (socket says 172.21.15.47:36522) 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069041700 1 -- 172.21.15.47:0/1322963391 learned_addr learned my addr 172.21.15.47:0/1322963391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069842700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa064102ac0 0x7fa064199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069041700 1 -- 172.21.15.47:0/1322963391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa064108a70 msgr2=0x7fa06419d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa06a043700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa064108a70 0x7fa06419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069041700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa064108a70 0x7fa06419d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069041700 1 -- 172.21.15.47:0/1322963391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa064102ac0 msgr2=0x7fa064199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069041700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa064102ac0 0x7fa064199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.854+0000 7fa069041700 1 -- 172.21.15.47:0/1322963391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06419e050 con 0x7fa064103480 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.855+0000 7fa069842700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa064102ac0 0x7fa064199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.855+0000 7fa06a043700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa064108a70 0x7fa06419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.855+0000 7fa069041700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064103480 0x7fa0641995d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c9b0 tx=0x7fa05400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.855+0000 7fa05affd700 1 -- 172.21.15.47:0/1322963391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054007b00 con 0x7fa064103480 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.855+0000 7fa05affd700 1 -- 172.21.15.47:0/1322963391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa054007ca0 con 0x7fa064103480 2024-04-25T21:10:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.855+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06419e340 con 0x7fa064103480 2024-04-25T21:10:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.855+0000 7fa05affd700 1 -- 172.21.15.47:0/1322963391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054018730 con 0x7fa064103480 2024-04-25T21:10:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.855+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0641a5bc0 con 0x7fa064103480 2024-04-25T21:10:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.857+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa064069730 con 0x7fa064103480 2024-04-25T21:10:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.857+0000 7fa05affd700 1 -- 172.21.15.47:0/1322963391 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa05400f450 con 0x7fa064103480 2024-04-25T21:10:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.858+0000 7fa05affd700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa0500a3db0 0x7fa0500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.858+0000 7fa069842700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa0500a3db0 0x7fa0500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.858+0000 7fa05affd700 1 -- 172.21.15.47:0/1322963391 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8764+0+0 (secure 0 0 0) 0x7fa0540d0990 con 0x7fa064103480 2024-04-25T21:10:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.858+0000 7fa069842700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa0500a3db0 0x7fa0500a6270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa060000c00 tx=0x7fa060005fb0 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:57.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:57.862+0000 7fa05affd700 1 -- 172.21.15.47:0/1322963391 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa054014030 con 0x7fa064103480 2024-04-25T21:10:58.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.070+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa06419a3a0 con 0x7fa064103480 2024-04-25T21:10:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.394+0000 7fa05affd700 1 -- 172.21.15.47:0/1322963391 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7fa05409cf40 con 0x7fa064103480 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa0500a3db0 msgr2=0x7fa0500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa0500a3db0 0x7fa0500a6270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa060000c00 tx=0x7fa060005fb0 comp rx=0 tx=0).stop 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064103480 msgr2=0x7fa0641995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064103480 0x7fa0641995d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c9b0 tx=0x7fa05400cd70 comp rx=0 tx=0).stop 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 shutdown_connections 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7fa0500a3db0 0x7fa0500a6270 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa060000c00 tx=0x7fa060005fb0 comp rx=0 tx=0).stop 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa064108a70 0x7fa06419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa064102ac0 0x7fa064199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 --2- 172.21.15.47:0/1322963391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064103480 0x7fa0641995d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:58.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.397+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 >> 172.21.15.47:0/1322963391 conn(0x7fa064079bb0 msgr2=0x7fa064101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:58.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.398+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 shutdown_connections 2024-04-25T21:10:58.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:10:58.398+0000 7fa06baa6700 1 -- 172.21.15.47:0/1322963391 wait complete. 2024-04-25T21:10:58.399 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-25T21:10:58.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:58 smithi047 ceph-mon[29195]: pgmap v341: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T21:10:58.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:58 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1322963391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:10:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:58 smithi186 ceph-mon[32529]: pgmap v341: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T21:10:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:58 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1322963391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:10:58.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:58 smithi092 ceph-mon[32524]: pgmap v341: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T21:10:58.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:58 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1322963391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:10:59.097 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T21:10:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:59.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:59 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/1322963391' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:10:59.666 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:10:59 smithi047 ceph-mon[29195]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:10:59.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:59 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/1322963391' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:10:59.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:10:59 smithi186 ceph-mon[32529]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:10:59.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:59 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/1322963391' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:10:59.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:10:59 smithi092 ceph-mon[32524]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:11:00.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:00 smithi047 ceph-mon[29195]: pgmap v343: 65 pgs: 65 active+clean; 55 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:11:00.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:00 smithi092 ceph-mon[32524]: pgmap v343: 65 pgs: 65 active+clean; 55 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:11:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:00 smithi186 ceph-mon[32529]: pgmap v343: 65 pgs: 65 active+clean; 55 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 -- 172.21.15.47:0/1606938698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c1026e0 msgr2=0x7f0f8c10e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/1606938698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c1026e0 0x7f0f8c10e770 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400b0a0 tx=0x7f0f8400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 -- 172.21.15.47:0/1606938698 shutdown_connections 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/1606938698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c1026e0 0x7f0f8c10e770 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/1606938698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c106990 0x7f0f8c106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/1606938698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8c101d20 0x7f0f8c1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 -- 172.21.15.47:0/1606938698 >> 172.21.15.47:0/1606938698 conn(0x7f0f8c0788f0 msgr2=0x7f0f8c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 -- 172.21.15.47:0/1606938698 shutdown_connections 2024-04-25T21:11:00.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.924+0000 7f0f93c0f700 1 -- 172.21.15.47:0/1606938698 wait complete. 2024-04-25T21:11:00.926 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:11:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:11:00.716+0000 7f1395bf5700 -1 client.15168: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T21:11:00.926 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:11:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x[29422]: 2024-04-25T21:11:00.717+0000 7f1395bf5700 -1 client.15168: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T21:11:00.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.925+0000 7f0f93c0f700 1 Processor -- start 2024-04-25T21:11:00.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.925+0000 7f0f93c0f700 1 -- start start 2024-04-25T21:11:00.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.925+0000 7f0f93c0f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c101d20 0x7f0f8c199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:00.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.925+0000 7f0f93c0f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c1026e0 0x7f0f8c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:00.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.925+0000 7f0f93c0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8c106990 0x7f0f8c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:00.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.925+0000 7f0f93c0f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c111ae0 con 0x7f0f8c101d20 2024-04-25T21:11:00.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.925+0000 7f0f93c0f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c111960 con 0x7f0f8c106990 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.925+0000 7f0f93c0f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c111c60 con 0x7f0f8c1026e0 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c101d20 0x7f0f8c199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c101d20 0x7f0f8c199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36534/0 (socket says 172.21.15.47:36534) 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 -- 172.21.15.47:0/726490836 learned_addr learned my addr 172.21.15.47:0/726490836 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f921ac700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8c106990 0x7f0f8c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f911aa700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c1026e0 0x7f0f8c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 -- 172.21.15.47:0/726490836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c1026e0 msgr2=0x7f0f8c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c1026e0 0x7f0f8c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 -- 172.21.15.47:0/726490836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8c106990 msgr2=0x7f0f8c19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8c106990 0x7f0f8c19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 -- 172.21.15.47:0/726490836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c19e080 con 0x7f0f8c101d20 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f921ac700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8c106990 0x7f0f8c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:00.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f911aa700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c1026e0 0x7f0f8c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:00.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f919ab700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c101d20 0x7f0f8c199070 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800d950 tx=0x7f0f8800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:00.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f82ffd700 1 -- 172.21.15.47:0/726490836 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88016070 con 0x7f0f8c101d20 2024-04-25T21:11:00.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f82ffd700 1 -- 172.21.15.47:0/726490836 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f88010460 con 0x7f0f8c101d20 2024-04-25T21:11:00.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8c19e370 con 0x7f0f8c101d20 2024-04-25T21:11:00.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.926+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8c1a5bf0 con 0x7f0f8c101d20 2024-04-25T21:11:00.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.927+0000 7f0f82ffd700 1 -- 172.21.15.47:0/726490836 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800f620 con 0x7f0f8c101d20 2024-04-25T21:11:00.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.927+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f70005330 con 0x7f0f8c101d20 2024-04-25T21:11:00.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.929+0000 7f0f82ffd700 1 -- 172.21.15.47:0/726490836 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0f88019070 con 0x7f0f8c101d20 2024-04-25T21:11:00.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.929+0000 7f0f82ffd700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f780a3e30 0x7f0f780a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:00.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.930+0000 7f0f911aa700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f780a3e30 0x7f0f780a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:00.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.930+0000 7f0f82ffd700 1 -- 172.21.15.47:0/726490836 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8239+0+0 (secure 0 0 0) 0x7f0f880d0650 con 0x7f0f8c101d20 2024-04-25T21:11:00.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.930+0000 7f0f911aa700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f780a3e30 0x7f0f780a62f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c078f60 tx=0x7f0f7c009260 comp rx=0 tx=0).ready entity=mgr.24811 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:00.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:00.933+0000 7f0f82ffd700 1 -- 172.21.15.47:0/726490836 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f8809cdb0 con 0x7f0f8c101d20 2024-04-25T21:11:01.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.143+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0f70005620 con 0x7f0f8c101d20 2024-04-25T21:11:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.413+0000 7f0f82ffd700 1 -- 172.21.15.47:0/726490836 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f0f8809c6b0 con 0x7f0f8c101d20 2024-04-25T21:11:01.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.417+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f780a3e30 msgr2=0x7f0f780a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:01.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.417+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f780a3e30 0x7f0f780a62f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c078f60 tx=0x7f0f7c009260 comp rx=0 tx=0).stop 2024-04-25T21:11:01.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.417+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c101d20 msgr2=0x7f0f8c199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:01.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.417+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c101d20 0x7f0f8c199070 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800d950 tx=0x7f0f8800dd10 comp rx=0 tx=0).stop 2024-04-25T21:11:01.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.418+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 shutdown_connections 2024-04-25T21:11:01.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.418+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:6800/957595556,v1:172.21.15.47:6801/957595556] conn(0x7f0f780a3e30 0x7f0f780a62f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:01.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.418+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f8c101d20 0x7f0f8c199070 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:01.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.418+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8c106990 0x7f0f8c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:01.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.418+0000 7f0f93c0f700 1 --2- 172.21.15.47:0/726490836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c1026e0 0x7f0f8c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:01.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.418+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 >> 172.21.15.47:0/726490836 conn(0x7f0f8c0788f0 msgr2=0x7f0f8c06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:01.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.418+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 shutdown_connections 2024-04-25T21:11:01.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-25T21:11:01.419+0000 7f0f93c0f700 1 -- 172.21.15.47:0/726490836 wait complete. 2024-04-25T21:11:01.420 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-25T21:11:01.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:01 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/726490836' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:11:01.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:01 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/726490836' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:11:01.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:01 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/726490836' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:11:02.100 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:11:02.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:11:02.156 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:11:02.189 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T21:11:02.189 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:11:02.206 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:11:02.222 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:11:02.246 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T21:11:02.247 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T21:11:02.247 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.a 2024-04-25T21:11:02.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:02 smithi047 systemd[1]: Stopping Ceph mon.a for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:02.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:02 smithi186 ceph-mon[32529]: pgmap v344: 65 pgs: 65 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:11:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:02 smithi186 ceph-mon[32529]: from='client.? 172.21.15.47:0/726490836' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:11:02.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:02 smithi186 ceph-mon[32529]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:11:02.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:02 smithi092 ceph-mon[32524]: pgmap v344: 65 pgs: 65 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:11:02.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:02 smithi092 ceph-mon[32524]: from='client.? 172.21.15.47:0/726490836' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:11:02.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:02 smithi092 ceph-mon[32524]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:11:02.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:02 smithi047 ceph-mon[29195]: pgmap v344: 65 pgs: 65 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:11:02.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:02 smithi047 ceph-mon[29195]: from='client.? 172.21.15.47:0/726490836' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:11:02.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:02 smithi047 ceph-mon[29195]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:11:02.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T21:11:02.547+0000 7f867a94f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:02.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T21:11:02.547+0000 7f867a94f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T21:11:03.073 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:02 smithi047 podman[113674]: 2024-04-25 21:11:02.794118906 +0000 UTC m=+0.354218890 container died 0248222cfd5e30e77fcbe0fa8e4c2e5fd27af1236219a26e29688b0346c64735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T21:11:03.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:03 smithi047 podman[113674]: 2024-04-25 21:11:03.173045486 +0000 UTC m=+0.733145462 container cleanup 0248222cfd5e30e77fcbe0fa8e4c2e5fd27af1236219a26e29688b0346c64735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T21:11:03.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 25 21:11:03 smithi047 bash[113674]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-a 2024-04-25T21:11:03.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T21:11:03.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:03.651 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T21:11:03.652 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T21:11:03.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.b 2024-04-25T21:11:03.958 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:03 smithi092 systemd[1]: Stopping Ceph mon.b for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:04.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-b[32520]: 2024-04-25T21:11:03.957+0000 7f84c052e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:04.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-b[32520]: 2024-04-25T21:11:03.957+0000 7f84c052e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T21:11:04.282 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:04 smithi092 podman[69002]: 2024-04-25 21:11:04.15124729 +0000 UTC m=+0.298550548 container died 68d25731cb93434d4b3ca64f5be5634701472d702d4f54792070758b8b0cfe4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T21:11:04.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:04 smithi092 podman[69002]: 2024-04-25 21:11:04.541108293 +0000 UTC m=+0.688411541 container cleanup 68d25731cb93434d4b3ca64f5be5634701472d702d4f54792070758b8b0cfe4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:11:04.782 INFO:journalctl@ceph.mon.b.smithi092.stdout:Apr 25 21:11:04 smithi092 bash[69002]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-b 2024-04-25T21:11:04.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T21:11:05.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:05.014 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T21:11:05.014 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T21:11:05.014 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.c 2024-04-25T21:11:05.309 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:05 smithi186 systemd[1]: Stopping Ceph mon.c for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:05.573 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-c[32502]: 2024-04-25T21:11:05.308+0000 7f57d093e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:05.574 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-c[32502]: 2024-04-25T21:11:05.308+0000 7f57d093e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T21:11:05.859 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:05 smithi186 podman[68995]: 2024-04-25 21:11:05.574306066 +0000 UTC m=+0.364280081 container died 7220ee8fe3f6109ae1d799d50819919eb4eaadf654baa479d17d8267b1fefccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T21:11:06.210 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:05 smithi186 podman[68995]: 2024-04-25 21:11:05.942465036 +0000 UTC m=+0.732439053 container cleanup 7220ee8fe3f6109ae1d799d50819919eb4eaadf654baa479d17d8267b1fefccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T21:11:06.210 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 25 21:11:05 smithi186 bash[68995]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mon-c 2024-04-25T21:11:06.355 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T21:11:06.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:06.402 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T21:11:06.402 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T21:11:06.402 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.x 2024-04-25T21:11:06.735 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:11:06 smithi047 systemd[1]: Stopping Ceph mgr.x for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:07.019 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:11:06 smithi047 podman[113795]: 2024-04-25 21:11:06.735459463 +0000 UTC m=+0.131753927 container died 9dba1d645dcfa0045ce2a63aa2366b63b8e210899e902c97756cecb337a47f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T21:11:07.360 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:11:07 smithi047 podman[113795]: 2024-04-25 21:11:07.094585316 +0000 UTC m=+0.490879769 container cleanup 9dba1d645dcfa0045ce2a63aa2366b63b8e210899e902c97756cecb337a47f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:11:07.360 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 25 21:11:07 smithi047 bash[113795]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-x 2024-04-25T21:11:07.583 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T21:11:07.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:07.618 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T21:11:07.618 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T21:11:07.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.y 2024-04-25T21:11:07.923 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:07 smithi092 systemd[1]: Stopping Ceph mgr.y for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:08.178 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:07 smithi092 podman[69125]: 2024-04-25 21:11:07.924438077 +0000 UTC m=+0.118539572 container died fa68289a9a105c5d8988eb6f7aba0b7755dae8a6bd62cd09d0ccf2a22742c30e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T21:11:08.429 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:08 smithi092 podman[69125]: 2024-04-25 21:11:08.262127996 +0000 UTC m=+0.456229487 container cleanup fa68289a9a105c5d8988eb6f7aba0b7755dae8a6bd62cd09d0ccf2a22742c30e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T21:11:08.429 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:08 smithi092 bash[69125]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y 2024-04-25T21:11:08.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T21:11:08.692 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:08 smithi092 podman[69139]: 2024-04-25 21:11:08.430148566 +0000 UTC m=+0.500805336 container remove fa68289a9a105c5d8988eb6f7aba0b7755dae8a6bd62cd09d0ccf2a22742c30e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T21:11:08.692 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:08 smithi092 systemd[1]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T21:11:08.692 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:08 smithi092 systemd[1]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-25T21:11:08.693 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:08 smithi092 systemd[1]: Stopped Ceph mgr.y for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T21:11:08.693 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Apr 25 21:11:08 smithi092 systemd[1]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.y.service: Consumed 20.161s CPU time. 2024-04-25T21:11:08.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:08.705 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T21:11:08.705 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T21:11:08.705 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.z 2024-04-25T21:11:09.034 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 21:11:08 smithi186 systemd[1]: Stopping Ceph mgr.z for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:09.322 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 21:11:09 smithi186 podman[69116]: 2024-04-25 21:11:09.034444485 +0000 UTC m=+0.135045007 container died 0bac3ee556bb1ca41b47ac20d63a718c48192a26dd4f5284a17bc2b3795a9bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T21:11:09.639 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 21:11:09 smithi186 podman[69116]: 2024-04-25 21:11:09.413786886 +0000 UTC m=+0.514387408 container cleanup 0bac3ee556bb1ca41b47ac20d63a718c48192a26dd4f5284a17bc2b3795a9bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T21:11:09.639 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 25 21:11:09 smithi186 bash[69116]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mgr-z 2024-04-25T21:11:09.788 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T21:11:09.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:09.825 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T21:11:09.826 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T21:11:09.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.0 2024-04-25T21:11:10.110 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:09 smithi047 systemd[1]: Stopping Ceph osd.0 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:10.384 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0[41378]: 2024-04-25T21:11:10.149+0000 7f6d17d13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:10.385 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0[41378]: 2024-04-25T21:11:10.149+0000 7f6d17d13700 -1 osd.0 103 *** Got signal Terminated *** 2024-04-25T21:11:10.385 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0[41378]: 2024-04-25T21:11:10.149+0000 7f6d17d13700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:11:15.900 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:15 smithi047 podman[113914]: 2024-04-25 21:11:15.624193197 +0000 UTC m=+5.597309863 container died 34a1b5c44213774ff974c7581b715550eb54e6d68077c536e320d7baccbd4fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T21:11:16.266 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:15 smithi047 podman[113914]: 2024-04-25 21:11:15.99963836 +0000 UTC m=+5.972755034 container cleanup 34a1b5c44213774ff974c7581b715550eb54e6d68077c536e320d7baccbd4fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T21:11:16.267 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:15 smithi047 bash[113914]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0 2024-04-25T21:11:16.609 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:16 smithi047 podman[113943]: 2024-04-25 21:11:16.267638218 +0000 UTC m=+0.641269022 container remove 34a1b5c44213774ff974c7581b715550eb54e6d68077c536e320d7baccbd4fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T21:11:16.860 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:16 smithi047 podman[114032]: 2024-04-25 21:11:16.610049009 +0000 UTC m=+0.095034810 container create 9b0ecefa4355c8a6ca07e35400233df9e516d5db2970c35dde41e5f9f65ce852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T21:11:16.860 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:16 smithi047 podman[114032]: 2024-04-25 21:11:16.533325757 +0000 UTC m=+0.018311558 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:11:17.170 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:16 smithi047 podman[114032]: 2024-04-25 21:11:16.862496875 +0000 UTC m=+0.347482687 container init 9b0ecefa4355c8a6ca07e35400233df9e516d5db2970c35dde41e5f9f65ce852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T21:11:17.170 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:16 smithi047 podman[114032]: 2024-04-25 21:11:16.867136129 +0000 UTC m=+0.352121942 container start 9b0ecefa4355c8a6ca07e35400233df9e516d5db2970c35dde41e5f9f65ce852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T21:11:17.170 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:16 smithi047 podman[114032]: 2024-04-25 21:11:16.954076046 +0000 UTC m=+0.439061847 container attach 9b0ecefa4355c8a6ca07e35400233df9e516d5db2970c35dde41e5f9f65ce852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:11:17.523 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 25 21:11:17 smithi047 podman[114032]: 2024-04-25 21:11:17.170777727 +0000 UTC m=+0.655763518 container died 9b0ecefa4355c8a6ca07e35400233df9e516d5db2970c35dde41e5f9f65ce852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T21:11:17.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T21:11:17.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:17.869 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T21:11:17.869 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T21:11:17.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.1 2024-04-25T21:11:18.165 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:17 smithi092 systemd[1]: Stopping Ceph osd.1 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:18.532 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1[37138]: 2024-04-25T21:11:18.164+0000 7f046f84d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:18.532 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1[37138]: 2024-04-25T21:11:18.164+0000 7f046f84d700 -1 osd.1 103 *** Got signal Terminated *** 2024-04-25T21:11:18.532 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1[37138]: 2024-04-25T21:11:18.164+0000 7f046f84d700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:11:23.916 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:23 smithi092 podman[69250]: 2024-04-25 21:11:23.615854947 +0000 UTC m=+5.552360223 container died eae8bb4dc5f789c58d8c2bda02c50d57baeea91eb836e47f82e74534f622cd25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:11:24.215 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:23 smithi092 podman[69250]: 2024-04-25 21:11:23.998411578 +0000 UTC m=+5.934916859 container cleanup eae8bb4dc5f789c58d8c2bda02c50d57baeea91eb836e47f82e74534f622cd25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T21:11:24.216 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:23 smithi092 bash[69250]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1 2024-04-25T21:11:24.531 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:24 smithi092 podman[69269]: 2024-04-25 21:11:24.2162961 +0000 UTC m=+0.603982657 container remove eae8bb4dc5f789c58d8c2bda02c50d57baeea91eb836e47f82e74534f622cd25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:11:24.861 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:24 smithi092 podman[69358]: 2024-04-25 21:11:24.440092404 +0000 UTC m=+0.018271650 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:11:24.861 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:24 smithi092 podman[69358]: 2024-04-25 21:11:24.567691652 +0000 UTC m=+0.145870915 container create 3cdc0346c07a66fd37befbe69c7548453a95970e684408600565326948aae3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T21:11:24.861 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:24 smithi092 podman[69358]: 2024-04-25 21:11:24.778385575 +0000 UTC m=+0.356564793 container init 3cdc0346c07a66fd37befbe69c7548453a95970e684408600565326948aae3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:11:24.861 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:24 smithi092 podman[69358]: 2024-04-25 21:11:24.781453109 +0000 UTC m=+0.359632343 container start 3cdc0346c07a66fd37befbe69c7548453a95970e684408600565326948aae3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:11:24.861 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:24 smithi092 podman[69358]: 2024-04-25 21:11:24.861766672 +0000 UTC m=+0.439945911 container attach 3cdc0346c07a66fd37befbe69c7548453a95970e684408600565326948aae3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T21:11:25.282 INFO:journalctl@ceph.osd.1.smithi092.stdout:Apr 25 21:11:25 smithi092 podman[69358]: 2024-04-25 21:11:25.074693673 +0000 UTC m=+0.652872902 container died 3cdc0346c07a66fd37befbe69c7548453a95970e684408600565326948aae3cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T21:11:25.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T21:11:25.713 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:25.713 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T21:11:25.713 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T21:11:25.713 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.2 2024-04-25T21:11:26.016 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:25 smithi186 systemd[1]: Stopping Ceph osd.2 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:26.469 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2[37727]: 2024-04-25T21:11:26.015+0000 7f71bb4a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:26.470 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2[37727]: 2024-04-25T21:11:26.015+0000 7f71bb4a7700 -1 osd.2 103 *** Got signal Terminated *** 2024-04-25T21:11:26.470 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2[37727]: 2024-04-25T21:11:26.015+0000 7f71bb4a7700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:11:31.592 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:31 smithi186 podman[69239]: 2024-04-25 21:11:31.342803881 +0000 UTC m=+5.440303600 container died 80c9c018eb4175e3258dc0c86c0b2059841cd7f92e2288f14b66a3f33f4b4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T21:11:31.850 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:31 smithi186 podman[69239]: 2024-04-25 21:11:31.666616204 +0000 UTC m=+5.764115928 container cleanup 80c9c018eb4175e3258dc0c86c0b2059841cd7f92e2288f14b66a3f33f4b4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T21:11:31.850 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:31 smithi186 bash[69239]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2 2024-04-25T21:11:32.220 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:31 smithi186 podman[69259]: 2024-04-25 21:11:31.851046362 +0000 UTC m=+0.503372934 container remove 80c9c018eb4175e3258dc0c86c0b2059841cd7f92e2288f14b66a3f33f4b4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T21:11:32.553 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:32 smithi186 podman[69353]: 2024-04-25 21:11:32.26864541 +0000 UTC m=+0.102809569 container create 29f34a58f30a7325e741f945fc051bda7150bbb3d0ef0ce5e4cb874617262592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T21:11:32.553 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:32 smithi186 podman[69353]: 2024-04-25 21:11:32.184133936 +0000 UTC m=+0.018298064 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:11:32.554 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:32 smithi186 podman[69353]: 2024-04-25 21:11:32.495855873 +0000 UTC m=+0.330019986 container init 29f34a58f30a7325e741f945fc051bda7150bbb3d0ef0ce5e4cb874617262592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T21:11:32.554 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:32 smithi186 podman[69353]: 2024-04-25 21:11:32.499293129 +0000 UTC m=+0.333457249 container start 29f34a58f30a7325e741f945fc051bda7150bbb3d0ef0ce5e4cb874617262592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:11:32.554 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:32 smithi186 podman[69353]: 2024-04-25 21:11:32.554116816 +0000 UTC m=+0.388280941 container attach 29f34a58f30a7325e741f945fc051bda7150bbb3d0ef0ce5e4cb874617262592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:11:32.970 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 25 21:11:32 smithi186 podman[69353]: 2024-04-25 21:11:32.796521606 +0000 UTC m=+0.630685720 container died 29f34a58f30a7325e741f945fc051bda7150bbb3d0ef0ce5e4cb874617262592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T21:11:33.361 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T21:11:33.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:33.400 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T21:11:33.400 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T21:11:33.400 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.3 2024-04-25T21:11:33.718 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:33 smithi047 systemd[1]: Stopping Ceph osd.3 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:34.110 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:33 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3[47878]: 2024-04-25T21:11:33.718+0000 7f2d8e661700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:34.110 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:33 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3[47878]: 2024-04-25T21:11:33.718+0000 7f2d8e661700 -1 osd.3 103 *** Got signal Terminated *** 2024-04-25T21:11:34.110 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:33 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3[47878]: 2024-04-25T21:11:33.718+0000 7f2d8e661700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:11:36.720 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:36 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:36.342+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:37.719 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:37.299+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:38.610 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:38 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3[47878]: 2024-04-25T21:11:38.259+0000 7f2d8ac55700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:13.859765+0000 front 2024-04-25T21:11:13.859631+0000 (oldest deadline 2024-04-25T21:11:37.359750+0000) 2024-04-25T21:11:38.719 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:38.254+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:38.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:38.496+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:38.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:38 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:38.367+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:39.360 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:39 smithi047 podman[114140]: 2024-04-25 21:11:39.089326038 +0000 UTC m=+5.483227488 container died b317f6556a8690350ef83592bb71208b42ae3c59b2a5633f2275de32f113cf1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T21:11:39.720 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:39.250+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:39.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:39.452+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:39.781 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:39 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:39.399+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:39.827 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:39 smithi047 podman[114140]: 2024-04-25 21:11:39.593297689 +0000 UTC m=+5.987199138 container cleanup b317f6556a8690350ef83592bb71208b42ae3c59b2a5633f2275de32f113cf1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:11:39.827 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:39 smithi047 bash[114140]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3 2024-04-25T21:11:40.110 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:39 smithi047 podman[114161]: 2024-04-25 21:11:39.827854846 +0000 UTC m=+0.736043147 container remove b317f6556a8690350ef83592bb71208b42ae3c59b2a5633f2275de32f113cf1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T21:11:40.422 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:40 smithi047 podman[114254]: 2024-04-25 21:11:40.128637107 +0000 UTC m=+0.112786855 container create cb2634195c21c2d3ca02864d004a417a95c35659fbde078f3109b2ce90522bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:11:40.423 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:40 smithi047 podman[114254]: 2024-04-25 21:11:40.034237779 +0000 UTC m=+0.018387508 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:11:40.423 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:40 smithi047 podman[114254]: 2024-04-25 21:11:40.331157443 +0000 UTC m=+0.315307146 container init cb2634195c21c2d3ca02864d004a417a95c35659fbde078f3109b2ce90522bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:11:40.423 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:40 smithi047 podman[114254]: 2024-04-25 21:11:40.334674395 +0000 UTC m=+0.318824102 container start cb2634195c21c2d3ca02864d004a417a95c35659fbde078f3109b2ce90522bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T21:11:40.423 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:40 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:40.140+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:40.501 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:40.221+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:40.781 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:40 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:40.356+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:40.860 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:40 smithi047 podman[114254]: 2024-04-25 21:11:40.422703178 +0000 UTC m=+0.406852899 container attach cb2634195c21c2d3ca02864d004a417a95c35659fbde078f3109b2ce90522bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T21:11:40.860 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 25 21:11:40 smithi047 podman[114254]: 2024-04-25 21:11:40.64318465 +0000 UTC m=+0.627334356 container died cb2634195c21c2d3ca02864d004a417a95c35659fbde078f3109b2ce90522bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T21:11:40.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:40.501+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:41.317 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:41 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:41.157+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:41.341 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T21:11:41.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:41.385 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T21:11:41.385 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T21:11:41.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.4 2024-04-25T21:11:41.469 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:41.215+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:41.700 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:41 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:41.352+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:41.700 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:41 smithi092 systemd[1]: Stopping Ceph osd.4 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:41.860 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:41 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:41.610+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:41.537+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:42.032 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:41 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4[43630]: 2024-04-25T21:11:41.699+0000 7f8efd063700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:42.032 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:41 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4[43630]: 2024-04-25T21:11:41.699+0000 7f8efd063700 -1 osd.4 103 *** Got signal Terminated *** 2024-04-25T21:11:42.032 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:41 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4[43630]: 2024-04-25T21:11:41.699+0000 7f8efd063700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:11:42.470 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:42 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:42.219+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:42.574 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:42 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:42.167+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:42.781 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:42 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:42.362+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:42.860 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:42 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:42.642+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:42.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:42 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:42.497+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:43.469 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:43 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:43.215+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:43.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:43 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:43.184+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:43.781 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:43 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:43.345+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:43.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:43 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:43.489+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:44.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:43 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:43.667+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:44.469 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:44 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:44.182+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:44.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:44.153+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:44.705 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:44 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:44.367+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:44.926 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:44 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:44.480+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:45.031 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:44 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4[43630]: 2024-04-25T21:11:44.911+0000 7f8ef9657700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.600869+0000 front 2024-04-25T21:11:22.600895+0000 (oldest deadline 2024-04-25T21:11:44.900974+0000) 2024-04-25T21:11:45.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:44 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:44.706+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:45.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:44.668+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:45.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:45 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:45.140+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:45.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:44 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:44.926+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:45.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:45 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:45.183+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:45.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:45 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:45.183+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:45.724 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:45 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:45.325+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:45.929 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:45 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:45.520+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:46.031 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:45 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4[43630]: 2024-04-25T21:11:45.933+0000 7f8ef9657700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.600869+0000 front 2024-04-25T21:11:22.600895+0000 (oldest deadline 2024-04-25T21:11:44.900974+0000) 2024-04-25T21:11:46.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:45 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:45.724+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:46.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:45 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:45.636+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:46.219 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:45 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:45.929+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:46.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:46 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:46.109+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:46.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:46 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:46.109+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:46.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:46 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:46.176+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:46.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:46 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:46.176+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:46.750 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:46 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:46.369+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:46.927 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:46 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:46.533+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:46.927 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:46 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:46.533+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:47.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:46 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:46.750+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:47.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:46 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:46.677+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:47.219 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:47 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:47.115+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:47.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:47 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:47.115+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:47.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:46 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:46.926+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:47.326 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:47 smithi092 podman[69477]: 2024-04-25 21:11:47.052785163 +0000 UTC m=+5.469467894 container died 1cee575c58392bd6d3f76076a6fc619abe993e3ee6141ab711d3ed411fedf704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T21:11:47.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:47 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:47.134+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:47.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:47 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:47.134+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:47.650 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:47 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:47.412+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:47.650 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:47 smithi092 podman[69477]: 2024-04-25 21:11:47.424813158 +0000 UTC m=+5.841495886 container cleanup 1cee575c58392bd6d3f76076a6fc619abe993e3ee6141ab711d3ed411fedf704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T21:11:47.650 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:47 smithi092 bash[69477]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4 2024-04-25T21:11:47.910 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:47 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:47.536+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:47.911 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:47 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:47.536+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:47.942 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:47 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:47.729+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:47.943 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:47 smithi092 podman[69494]: 2024-04-25 21:11:47.651008833 +0000 UTC m=+0.600214709 container remove 1cee575c58392bd6d3f76076a6fc619abe993e3ee6141ab711d3ed411fedf704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:11:48.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:47 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:47.701+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:48.219 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:48 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:48.120+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:48.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:48 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:48.120+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:48.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:47 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:47.910+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:48.261 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:47 smithi092 podman[69586]: 2024-04-25 21:11:47.846306938 +0000 UTC m=+0.017827734 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:11:48.262 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:47 smithi092 podman[69586]: 2024-04-25 21:11:47.968618369 +0000 UTC m=+0.140139210 container create ce37ed887a5608ac057b0415a4d1443ec9f28ea5930382f026fb1bfc5863ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T21:11:48.262 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:48 smithi092 podman[69586]: 2024-04-25 21:11:48.187494396 +0000 UTC m=+0.359015201 container init ce37ed887a5608ac057b0415a4d1443ec9f28ea5930382f026fb1bfc5863ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T21:11:48.262 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:48 smithi092 podman[69586]: 2024-04-25 21:11:48.190603641 +0000 UTC m=+0.362124437 container start ce37ed887a5608ac057b0415a4d1443ec9f28ea5930382f026fb1bfc5863ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T21:11:48.262 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:48 smithi092 podman[69586]: 2024-04-25 21:11:48.262465742 +0000 UTC m=+0.433986542 container attach ce37ed887a5608ac057b0415a4d1443ec9f28ea5930382f026fb1bfc5863ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T21:11:48.532 INFO:journalctl@ceph.osd.4.smithi092.stdout:Apr 25 21:11:48 smithi092 podman[69586]: 2024-04-25 21:11:48.494746687 +0000 UTC m=+0.666267486 container died ce37ed887a5608ac057b0415a4d1443ec9f28ea5930382f026fb1bfc5863ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T21:11:48.532 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:48 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:48.408+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:48.532 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:48 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:48.408+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:48.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:48 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:48.130+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:48.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:48 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:48.130+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:48.865 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:48 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:48.564+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:48.865 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:48 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:48.564+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:49.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:48 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:48.709+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:49.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T21:11:49.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:48 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:48.666+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:49.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:49.134 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T21:11:49.134 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T21:11:49.134 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.5 2024-04-25T21:11:49.144 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:48 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:48.865+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:49.426 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:49 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:49.144+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:49.426 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:49 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:49.144+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:49.426 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:49 smithi186 systemd[1]: Stopping Ceph osd.5 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:49.426 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:49 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:49.425+0000 7feaf6e93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:49.426 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:49 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:49.425+0000 7feaf6e93700 -1 osd.5 103 *** Got signal Terminated *** 2024-04-25T21:11:49.426 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:49 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:49.425+0000 7feaf6e93700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:11:49.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:49 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:49.174+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:49.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:49 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:49.174+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:49.719 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:49 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:49.570+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:49.719 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:49 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:49.570+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:49.758 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:49 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:49.407+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:49.759 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:49 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:49.407+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:50.031 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:49 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:49.758+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:50.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:49 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:49.694+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:50.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:49 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:49.694+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:50.170 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:49 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:49.876+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:50.469 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:50 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:50.170+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:50.470 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:50 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:50.170+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:50.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:50 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:50.126+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:50.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:50 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:50.126+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:50.739 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:50 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:50.371+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:50.739 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:50 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:50.371+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:50.922 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:50 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:50.580+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:50.923 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:50 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:50.580+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:51.031 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:50 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:50.739+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:51.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:50 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:50.674+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:51.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:50 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:50.674+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:51.185 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:50 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:50.922+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:51.469 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:51.185+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:51.469 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:51.185+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:51.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:51.147+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:51.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:51.147+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:51.728 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:51 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:51.374+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:51.728 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:51 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:51.374+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:51.875 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:51.614+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:51.875 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:51.614+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:52.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:51 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:51.728+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:52.061 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:51.691+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:52.061 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:51.691+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:52.173 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:51.875+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:52.173 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:51 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:51.875+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:11:52.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:52.188+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:52.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:52.188+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:52.469 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:52.173+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:52.469 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:52.173+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:52.708 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:52.406+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:52.708 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:52.406+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:52.708 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:52.406+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:11:52.919 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:52.659+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:52.920 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:52.659+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:53.031 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:52 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:52.707+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:53.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:52.736+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:53.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:52 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:52.736+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:53.176 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:52.919+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:53.176 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:52 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:52.919+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:11:53.470 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:53.176+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:53.470 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:53.176+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:53.470 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:53.176+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:28.265252+0000 front 2024-04-25T21:11:28.265229+0000 (oldest deadline 2024-04-25T21:11:52.365346+0000) 2024-04-25T21:11:53.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:53 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:53.171+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:53.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:53 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:53.171+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:53.714 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:53 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:53.418+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:53.714 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:53 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:53.418+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:53.714 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:53 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:53.418+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:11:53.965 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:53.649+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:53.965 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:53.649+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:54.031 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:53 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:53.714+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:54.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:53 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:53.714+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:11:54.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:53 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:53.764+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:54.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:53 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:53.764+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:54.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:53 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:53.764+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:11:54.219 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:53.965+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:54.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:53 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:53.965+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:11:54.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:54.210+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:10.063192+0000 front 2024-04-25T21:11:10.063337+0000 (oldest deadline 2024-04-25T21:11:35.963085+0000) 2024-04-25T21:11:54.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:54.210+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.264396+0000 front 2024-04-25T21:11:21.264444+0000 (oldest deadline 2024-04-25T21:11:45.964464+0000) 2024-04-25T21:11:54.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-25T21:11:54.210+0000 7feaef47f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:28.265252+0000 front 2024-04-25T21:11:28.265229+0000 (oldest deadline 2024-04-25T21:11:52.365346+0000) 2024-04-25T21:11:54.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:54.125+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:54.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:54.125+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:54.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:54.125+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:11:54.703 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:54.453+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:54.704 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:54.453+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:54.704 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:54.453+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:11:54.919 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:54.666+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:54.919 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:54.666+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:55.031 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:54.703+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:55.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:54 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:54.703+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:11:55.082 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:54.788+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:55.083 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:54.788+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:55.083 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:54 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:54.788+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:11:55.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:54 smithi186 podman[69470]: 2024-04-25 21:11:54.919741245 +0000 UTC m=+5.591930579 container died 287478ade75a23f28a0262dde776ab6fbdd66a21385e2e863a22dc1d7398454c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T21:11:55.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:54.946+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:55.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:54 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:54.946+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:11:55.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:55.083+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:55.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:55.083+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:55.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:55.083+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:11:55.700 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:55 smithi186 podman[69470]: 2024-04-25 21:11:55.250491433 +0000 UTC m=+5.922680774 container cleanup 287478ade75a23f28a0262dde776ab6fbdd66a21385e2e863a22dc1d7398454c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T21:11:55.701 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:55 smithi186 bash[69470]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5 2024-04-25T21:11:55.701 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:55 smithi186 podman[69488]: 2024-04-25 21:11:55.46021426 +0000 UTC m=+0.535267303 container remove 287478ade75a23f28a0262dde776ab6fbdd66a21385e2e863a22dc1d7398454c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T21:11:55.781 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:55 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:55.703+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:55.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:55 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:55.703+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:11:55.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:55 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:55.476+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:55.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:55 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:55.476+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:55.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:55 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:55.476+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:11:55.958 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:55 smithi186 podman[69578]: 2024-04-25 21:11:55.877708024 +0000 UTC m=+0.113643660 container create 9c0b031bd86c003cb02086ff0c49a5d9bebebd43f7c4280e7d3b5a0b881ad9c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T21:11:55.958 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:55 smithi186 podman[69578]: 2024-04-25 21:11:55.782137923 +0000 UTC m=+0.018073569 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:11:55.959 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:55.700+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:55.959 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:55.700+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:55.959 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:55.700+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:11:56.043 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:55.776+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:56.043 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:55.776+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:56.043 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:55 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:55.776+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:11:56.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:55.986+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:56.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:55 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:55.986+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:11:56.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:56 smithi186 podman[69578]: 2024-04-25 21:11:56.088753209 +0000 UTC m=+0.324688841 container init 9c0b031bd86c003cb02086ff0c49a5d9bebebd43f7c4280e7d3b5a0b881ad9c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T21:11:56.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:56 smithi186 podman[69578]: 2024-04-25 21:11:56.092006597 +0000 UTC m=+0.327942231 container start 9c0b031bd86c003cb02086ff0c49a5d9bebebd43f7c4280e7d3b5a0b881ad9c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T21:11:56.220 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:56 smithi186 podman[69578]: 2024-04-25 21:11:56.17196333 +0000 UTC m=+0.407898979 container attach 9c0b031bd86c003cb02086ff0c49a5d9bebebd43f7c4280e7d3b5a0b881ad9c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T21:11:56.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:56.043+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:56.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:56.043+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:56.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:56.043+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:11:56.654 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:56 smithi186 podman[69578]: 2024-04-25 21:11:56.389112442 +0000 UTC m=+0.625048069 container died 9c0b031bd86c003cb02086ff0c49a5d9bebebd43f7c4280e7d3b5a0b881ad9c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:11:56.781 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:56.669+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:56.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:56.669+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:11:56.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:56.464+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:56.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:56.464+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:56.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:56 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:56.464+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:11:56.912 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 25 21:11:56 smithi186 podman[69604]: 2024-04-25 21:11:56.900249991 +0000 UTC m=+0.505680480 container remove 9c0b031bd86c003cb02086ff0c49a5d9bebebd43f7c4280e7d3b5a0b881ad9c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T21:11:56.912 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:56 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:56.654+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:56.912 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:56 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:56.654+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:56.913 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:56 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:56.654+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:11:56.917 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T21:11:56.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:56.996 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T21:11:56.996 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T21:11:56.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.6 2024-04-25T21:11:57.028 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:56.778+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:57.028 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:56.778+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:57.028 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:56.778+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:11:57.219 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:57.007+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:57.220 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:57.007+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:11:57.303 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:57 smithi047 systemd[1]: Stopping Ceph osd.6 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:11:57.303 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:57.067+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:57.303 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:57.067+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:57.303 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:57.067+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:11:57.303 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:57.303+0000 7f14259dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:57.303 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:57.303+0000 7f14259dc700 -1 osd.6 103 *** Got signal Terminated *** 2024-04-25T21:11:57.304 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:57.303+0000 7f14259dc700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:11:57.781 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:57.625+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:57.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:57.625+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:11:57.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:57.433+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:57.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:57.433+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:57.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:57 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:57.433+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:11:57.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:57.622+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:57.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:57.622+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:57.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:57 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:57.622+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:11:58.073 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:57.737+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:58.073 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:57.737+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:58.074 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:57 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:57.737+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:11:58.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:58.074+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:58.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:58.074+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:58.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:58.074+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:11:58.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:58.045+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:58.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:58.045+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:11:58.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:58.634+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:58.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:58.634+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:11:58.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:58.415+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:58.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:58.415+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:58.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:58 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:58.415+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:11:58.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:58.650+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:58.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:58.650+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:58.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:58 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:58.650+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:11:59.053 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:58.765+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:11:59.053 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:58.765+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:11:59.053 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:58 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:58.765+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:11:59.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:59.052+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:11:59.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:59.052+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:11:59.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:11:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:11:59.052+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:11:59.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:59 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:59.045+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:11:59.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:11:59 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:11:59.045+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:11:59.781 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:59.639+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:11:59.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:11:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:11:59.639+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:11:59.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:59.452+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:11:59.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:59.452+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:11:59.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:11:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:11:59.452+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:11:59.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:59 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:59.689+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:11:59.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:59 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:59.689+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:11:59.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:11:59 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:11:59.689+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:00.048 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:59.776+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:00.048 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:59.776+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:00.048 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:59.776+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:00.048 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:11:59 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:11:59.776+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:00.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:00.047+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:12:00.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:00.047+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:12:00.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:00.047+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:12:00.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:00.025+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:00.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:00.025+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:00.781 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:00.616+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:00.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:00.616+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:00.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:00.444+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:00.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:00.444+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:00.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:00 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:00.444+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:00.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:00.681+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:00.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:00.681+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:00.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:00 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:00.681+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:01.076 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:00.731+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:01.076 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:00.731+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:01.076 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:00.731+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:01.076 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:00 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:00.731+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:01.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:01.075+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:12:01.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:01.075+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:12:01.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:01.075+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:12:01.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:01 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:01.045+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:01.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:01 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:01.045+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:01.781 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:01.659+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:01.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:01.659+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:01.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:01.479+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:01.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:01.479+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:01.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:01.479+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:01.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:01 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:01.633+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:01.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:01 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:01.633+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:01.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:01 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:01.633+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:02.065 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:01.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:02.066 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:01.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:02.066 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:01.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:02.066 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:01 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:01.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:02.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:02.065+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.762895+0000 front 2024-04-25T21:11:14.763085+0000 (oldest deadline 2024-04-25T21:11:40.063135+0000) 2024-04-25T21:12:02.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:02.065+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.063919+0000 front 2024-04-25T21:11:20.064005+0000 (oldest deadline 2024-04-25T21:11:44.763729+0000) 2024-04-25T21:12:02.360 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6[54855]: 2024-04-25T21:12:02.065+0000 7f141dfc8700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.665062+0000 front 2024-04-25T21:11:30.665093+0000 (oldest deadline 2024-04-25T21:11:54.065409+0000) 2024-04-25T21:12:02.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:02 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:02.048+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:02.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:02 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:02.048+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:02.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:02 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:02.048+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:02.752 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:02 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:02.675+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:02.753 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:02 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:02.675+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:02.753 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:02 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:02.675+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:02.753 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:02 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:02.675+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:02.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:02.644+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:02.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:02.644+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:02.782 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:02.644+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.670147+0000 front 2024-04-25T21:11:37.669885+0000 (oldest deadline 2024-04-25T21:12:01.769890+0000) 2024-04-25T21:12:02.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:02.521+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:02.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:02.521+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:02.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:02 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:02.521+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:02.914 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:02 smithi047 podman[114370]: 2024-04-25 21:12:02.647730074 +0000 UTC m=+5.448845987 container died b796941df15aa6f5bcd76e90c542f88662e23f7dd37af145aab7ff0ae211dfa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T21:12:02.914 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:02.822+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:02.914 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:02.822+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:02.914 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:02.822+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:02.914 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:02 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:02.822+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:03.278 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:03 smithi047 podman[114370]: 2024-04-25 21:12:03.011603872 +0000 UTC m=+5.812719784 container cleanup b796941df15aa6f5bcd76e90c542f88662e23f7dd37af145aab7ff0ae211dfa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T21:12:03.278 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:03 smithi047 bash[114370]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6 2024-04-25T21:12:03.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:03 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:03.020+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:03.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:03 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:03.020+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:03.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:03 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:03.020+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:03.610 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:03 smithi047 podman[114387]: 2024-04-25 21:12:03.279080541 +0000 UTC m=+0.628273263 container remove b796941df15aa6f5bcd76e90c542f88662e23f7dd37af145aab7ff0ae211dfa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T21:12:03.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:03 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:03.681+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:03.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:03 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:03.681+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:03.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:03 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:03.681+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:03.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:03 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:03.681+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:03.972 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:03 smithi047 podman[114480]: 2024-04-25 21:12:03.638323165 +0000 UTC m=+0.110033239 container create ccf2727f191a59e73fa0bc5ca5d08738c58674514cec36d41d5caca692a80e8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T21:12:03.972 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:03 smithi047 podman[114480]: 2024-04-25 21:12:03.54675562 +0000 UTC m=+0.018465718 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:12:03.973 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:03.802+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:03.973 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:03.802+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:03.973 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:03.802+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:03.973 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:03 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:03.802+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:04.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:03.665+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:04.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:03.665+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:04.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:03.665+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.670147+0000 front 2024-04-25T21:11:37.669885+0000 (oldest deadline 2024-04-25T21:12:01.769890+0000) 2024-04-25T21:12:04.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:03.567+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:04.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:03.568+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:04.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:03.568+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:04.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:03 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:03.568+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:04.280 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:03 smithi047 podman[114480]: 2024-04-25 21:12:03.973698184 +0000 UTC m=+0.445408255 container init ccf2727f191a59e73fa0bc5ca5d08738c58674514cec36d41d5caca692a80e8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T21:12:04.280 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:03 smithi047 podman[114480]: 2024-04-25 21:12:03.977391285 +0000 UTC m=+0.449101359 container start ccf2727f191a59e73fa0bc5ca5d08738c58674514cec36d41d5caca692a80e8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T21:12:04.281 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:04 smithi047 podman[114480]: 2024-04-25 21:12:04.107484509 +0000 UTC m=+0.579194599 container attach ccf2727f191a59e73fa0bc5ca5d08738c58674514cec36d41d5caca692a80e8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T21:12:04.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:04 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:04.052+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:04.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:04 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:04.052+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:04.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:04 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:04.052+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:04.562 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 25 21:12:04 smithi047 podman[114480]: 2024-04-25 21:12:04.281913775 +0000 UTC m=+0.753623857 container died ccf2727f191a59e73fa0bc5ca5d08738c58674514cec36d41d5caca692a80e8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True) 2024-04-25T21:12:04.818 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:04.787+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:04.818 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:04.787+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:04.818 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:04.787+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:04.818 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:04 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:04.787+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:04.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T21:12:04.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:04.915 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T21:12:04.915 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T21:12:04.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.7 2024-04-25T21:12:04.941 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:04.613+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:04.942 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:04.613+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:04.942 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:04.613+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:04.942 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:04.613+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:04.942 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:04.662+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:04.942 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:04.662+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:04.942 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:04 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:04.662+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.670147+0000 front 2024-04-25T21:11:37.669885+0000 (oldest deadline 2024-04-25T21:12:01.769890+0000) 2024-04-25T21:12:04.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:04 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:04.660+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:04.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:04 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:04.660+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:04.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:04 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:04.660+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:04.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:04 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:04.660+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:05.200 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:04 smithi092 systemd[1]: Stopping Ceph osd.7 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:05.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:05.042+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:05.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:05.042+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:05.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:05.042+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:05.531 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:05.200+0000 7ff497bc3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:05.531 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:05.200+0000 7ff497bc3700 -1 osd.7 103 *** Got signal Terminated *** 2024-04-25T21:12:05.531 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:05.200+0000 7ff497bc3700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:12:05.860 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:05 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:05.764+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:05.860 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:05 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:05.764+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:05.860 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:05 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:05.764+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:05.860 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:05 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:05.764+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:05.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:05.662+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:05.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:05.662+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:05.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:05.662+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:05.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:05 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:05.662+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:06.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:05.669+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:06.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:05.669+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:06.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:05.669+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.670147+0000 front 2024-04-25T21:11:37.669885+0000 (oldest deadline 2024-04-25T21:12:01.769890+0000) 2024-04-25T21:12:06.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:05.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:06.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:05.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:06.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:05.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:06.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:05.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:06.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:06 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:06.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:06.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:06 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:06.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:06.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:06 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:06.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:06.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:06 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:06.640+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:06.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:06 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:06.640+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:06.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:06 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:06.640+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:06.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:06 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:06.640+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:07.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:06 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:06.684+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:07.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:06 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:06.684+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:07.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:06 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:06.684+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.670147+0000 front 2024-04-25T21:11:37.669885+0000 (oldest deadline 2024-04-25T21:12:01.769890+0000) 2024-04-25T21:12:07.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:06 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:06.607+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:07.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:06 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:06.607+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:07.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:06 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:06.607+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:07.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:06 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:06.607+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:07.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:06 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:06.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:07.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:06 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:06.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:07.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:06 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:06.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:07.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:06 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:06.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:07.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:07 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:07.129+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:07.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:07 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:07.129+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:07.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:07 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:07.129+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:07.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:07 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:07.598+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:07.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:07 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:07.598+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:07.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:07 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:07.598+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:07.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:07 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:07.598+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:08.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:07 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:07.698+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:08.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:07 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:07.698+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:08.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:07 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:07.698+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.670147+0000 front 2024-04-25T21:11:37.669885+0000 (oldest deadline 2024-04-25T21:12:01.769890+0000) 2024-04-25T21:12:08.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:07 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:07.698+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:41.770366+0000 front 2024-04-25T21:11:41.770385+0000 (oldest deadline 2024-04-25T21:12:07.670525+0000) 2024-04-25T21:12:08.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:07 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:07.613+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:08.033 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:07 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:07.613+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:08.033 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:07 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:07.613+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:08.033 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:07 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:07.613+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:08.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:07 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:07.762+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:08.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:07 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:07.762+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:08.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:07 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:07.762+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:08.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:07 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:07.762+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:08.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:08.088+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:08.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:08.088+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:08.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:08.088+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:08.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:08.581+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:08.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:08.581+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:08.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:08.581+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:08.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:08.581+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:08.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:08 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:08.581+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:09.031 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:08.734+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:09.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:08.734+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:09.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:08.734+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.670147+0000 front 2024-04-25T21:11:37.669885+0000 (oldest deadline 2024-04-25T21:12:01.769890+0000) 2024-04-25T21:12:09.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:08.734+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:41.770366+0000 front 2024-04-25T21:11:41.770385+0000 (oldest deadline 2024-04-25T21:12:07.670525+0000) 2024-04-25T21:12:09.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:08.658+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:09.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:08.658+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:09.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:08.658+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:09.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:08.658+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:09.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:08 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:08.658+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:09.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:08 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:08.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:09.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:08 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:08.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:09.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:08 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:08.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:09.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:08 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:08.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:09.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:08 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:08.780+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:09.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:09 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:09.062+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:09.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:09 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:09.062+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:09.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:09 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:09.062+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:09.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:09 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:09.585+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:09.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:09 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:09.585+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:09.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:09 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:09.585+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:09.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:09 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:09.585+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:09.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:09 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:09.585+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:10.031 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:09.752+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.666753+0000 front 2024-04-25T21:11:22.666777+0000 (oldest deadline 2024-04-25T21:11:43.767011+0000) 2024-04-25T21:12:10.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:09.752+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.768635+0000 front 2024-04-25T21:11:30.768554+0000 (oldest deadline 2024-04-25T21:11:53.668609+0000) 2024-04-25T21:12:10.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:09.752+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.670147+0000 front 2024-04-25T21:11:37.669885+0000 (oldest deadline 2024-04-25T21:12:01.769890+0000) 2024-04-25T21:12:10.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7[50174]: 2024-04-25T21:12:09.752+0000 7ff4941b7700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:41.770366+0000 front 2024-04-25T21:11:41.770385+0000 (oldest deadline 2024-04-25T21:12:07.670525+0000) 2024-04-25T21:12:10.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:09.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:10.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:09.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:10.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:09.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:10.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:09.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:10.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:09.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:10.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:09 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:09.801+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:10.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:09 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:09.801+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:10.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:09 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:09.801+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:10.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:09 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:09.801+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:10.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:09 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:09.801+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:10.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:10 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:10.069+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:10.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:10 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:10.069+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:10.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:10 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:10.069+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:10.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:10 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:10.549+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:10.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:10 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:10.549+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:10.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:10 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:10.549+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:10.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:10 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:10.549+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:10.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:10 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:10.549+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:11.032 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:10 smithi092 podman[69693]: 2024-04-25 21:12:10.74747057 +0000 UTC m=+5.640982626 container died 747946a74be53d02eb4818622ae7e211656023be086224992cdcd86fd687bcf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:12:11.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:10 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:10.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:11.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:10 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:10.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:11.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:10 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:10.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:11.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:10 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:10.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:11.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:10 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:10.694+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:11.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:10.786+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:11.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:10.786+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:11.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:10.786+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:11.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:10.786+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:11.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:10 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:10.786+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:11.367 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:11 smithi092 podman[69693]: 2024-04-25 21:12:11.15910014 +0000 UTC m=+6.052612202 container cleanup 747946a74be53d02eb4818622ae7e211656023be086224992cdcd86fd687bcf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T21:12:11.367 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:11 smithi092 bash[69693]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7 2024-04-25T21:12:11.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:11 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:11.030+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:11.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:11 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:11.030+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:11.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:11 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:11.030+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:11.683 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:11 smithi092 podman[69722]: 2024-04-25 21:12:11.368307886 +0000 UTC m=+0.618548087 container remove 747946a74be53d02eb4818622ae7e211656023be086224992cdcd86fd687bcf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T21:12:11.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:11 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:11.596+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:11.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:11 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:11.596+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:11.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:11 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:11.596+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:11.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:11 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:11.596+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:11.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:11 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:11.596+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:12.012 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:11 smithi092 podman[69817]: 2024-04-25 21:12:11.686325445 +0000 UTC m=+0.018050171 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:12:12.012 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:11 smithi092 podman[69817]: 2024-04-25 21:12:11.802586362 +0000 UTC m=+0.134311120 container create 14dc26631a0c82bcbbb502117fde559d144e91f0e1235a1d40cefc222ee082d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:12:12.012 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:12 smithi092 podman[69817]: 2024-04-25 21:12:12.013236745 +0000 UTC m=+0.344961457 container init 14dc26631a0c82bcbbb502117fde559d144e91f0e1235a1d40cefc222ee082d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T21:12:12.013 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:11 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:11.683+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:12.013 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:11 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:11.683+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:12.013 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:11 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:11.683+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:12.013 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:11 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:11.683+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:12.013 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:11 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:11.683+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:12.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:11 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:11.806+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:12.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:11 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:11.806+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:12.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:11 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:11.806+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:12.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:11 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:11.806+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:12.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:11 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:11.806+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:12.281 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:12 smithi092 podman[69817]: 2024-04-25 21:12:12.016580018 +0000 UTC m=+0.348304730 container start 14dc26631a0c82bcbbb502117fde559d144e91f0e1235a1d40cefc222ee082d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T21:12:12.281 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:12 smithi092 podman[69817]: 2024-04-25 21:12:12.08817472 +0000 UTC m=+0.419899435 container attach 14dc26631a0c82bcbbb502117fde559d144e91f0e1235a1d40cefc222ee082d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T21:12:12.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:12.038+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:12.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:12.038+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:12.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:12.038+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:12.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:12.038+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:46.242973+0000 front 2024-04-25T21:11:46.242998+0000 (oldest deadline 2024-04-25T21:12:11.542885+0000) 2024-04-25T21:12:12.598 INFO:journalctl@ceph.osd.7.smithi092.stdout:Apr 25 21:12:12 smithi092 podman[69817]: 2024-04-25 21:12:12.305097007 +0000 UTC m=+0.636821720 container died 14dc26631a0c82bcbbb502117fde559d144e91f0e1235a1d40cefc222ee082d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T21:12:12.883 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:12 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:12.703+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:12.883 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:12 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:12.703+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:12.883 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:12 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:12.703+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:12.883 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:12 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:12.703+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:12.883 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:12 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:12.703+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:12.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T21:12:12.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:12.599+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:12.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:12.599+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:12.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:12.599+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:12.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:12.599+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:12.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:12 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:12.599+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:12.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:12.995 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T21:12:12.995 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T21:12:12.995 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.8 2024-04-25T21:12:13.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:12.758+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:13.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:12.758+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:13.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:12.758+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:13.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:12.758+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:13.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:12 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:12.758+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:13.327 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:13 smithi186 systemd[1]: Stopping Ceph osd.8 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:13.327 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:13.078+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:13.327 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:13.078+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:13.327 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:13.078+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:13.327 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:13.078+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:46.242973+0000 front 2024-04-25T21:11:46.242998+0000 (oldest deadline 2024-04-25T21:12:11.542885+0000) 2024-04-25T21:12:13.327 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:13.327+0000 7fc51e528700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:13.327 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:13.327+0000 7fc51e528700 -1 osd.8 103 *** Got signal Terminated *** 2024-04-25T21:12:13.327 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:13.327+0000 7fc51e528700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:12:13.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:13.597+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:13.971 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:13.597+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:13.971 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:13.597+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:13.971 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:13.597+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:13.971 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:13 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:13.597+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:14.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:13 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:13.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:14.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:13 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:13.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:14.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:13 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:13.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:14.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:13 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:13.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:14.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:13 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:13.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:14.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:13 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:13.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:14.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:13 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:13.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:14.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:13 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:13.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:14.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:13 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:13.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:14.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:13 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:13.784+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:14.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:14.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:14.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:14.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:14.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:14.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:14.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:14.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:46.242973+0000 front 2024-04-25T21:11:46.242998+0000 (oldest deadline 2024-04-25T21:12:11.542885+0000) 2024-04-25T21:12:14.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:14.562+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:14.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:14.562+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:14.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:14.562+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:14.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:14.562+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:14.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:14.562+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:15.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:14 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:14.689+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:15.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:14 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:14.689+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:15.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:14 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:14.689+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:15.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:14 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:14.689+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:15.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:14 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:14.689+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:15.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:14.818+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:15.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:14.818+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:15.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:14.818+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:15.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:14.818+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:15.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:14 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:14.818+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:15.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:15.108+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:15.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:15.108+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:15.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:15.108+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:15.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:15.108+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:46.242973+0000 front 2024-04-25T21:11:46.242998+0000 (oldest deadline 2024-04-25T21:12:11.542885+0000) 2024-04-25T21:12:15.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:15.526+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:15.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:15.526+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:15.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:15.526+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:15.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:15.526+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:15.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:15 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:15.526+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:16.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:15 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:15.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:16.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:15 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:15.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:16.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:15 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:15.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:16.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:15 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:15.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:16.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:15 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:15.673+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:16.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:15.837+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:16.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:15.837+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:16.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:15.837+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:16.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:15.837+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:16.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:15.837+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:16.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:15 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:15.837+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:16.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:16.132+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:16.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:16.132+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:16.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:16.132+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:16.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:16.132+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:46.242973+0000 front 2024-04-25T21:11:46.242998+0000 (oldest deadline 2024-04-25T21:12:11.542885+0000) 2024-04-25T21:12:16.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:16.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:16.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:16.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:16.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:16.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:16.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:16.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:16.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:16.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:16.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:16 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:16.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:17.034 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:16 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:16.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:17.034 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:16 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:16.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:17.034 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:16 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:16.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:17.034 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:16 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:16.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:17.034 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:16 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:16.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:17.034 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:16 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:16.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:17.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:16.856+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:17.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:16.856+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:17.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:16.856+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:17.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:16.856+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:17.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:16.856+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:17.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:16 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:16.856+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:17.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:17.134+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:17.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:17.134+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:17.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:17.134+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:17.470 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:17.134+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:46.242973+0000 front 2024-04-25T21:11:46.242998+0000 (oldest deadline 2024-04-25T21:12:11.542885+0000) 2024-04-25T21:12:17.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:17.496+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:17.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:17.496+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:17.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:17.496+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:17.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:17.496+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:17.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:17.496+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:17.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:17.496+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:18.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:17 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:17.644+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:18.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:17 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:17.644+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:18.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:17 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:17.645+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:18.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:17 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:17.645+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:18.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:17 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:17.645+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:18.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:17 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:17.645+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:18.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:17.878+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:18.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:17.878+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:18.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:17.878+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:18.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:17.878+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:18.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:17.878+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:18.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:17 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:17.878+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:18.471 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:18.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:20.836712+0000 front 2024-04-25T21:11:20.836760+0000 (oldest deadline 2024-04-25T21:11:44.336897+0000) 2024-04-25T21:12:18.471 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:18.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.138896+0000 front 2024-04-25T21:11:30.138926+0000 (oldest deadline 2024-04-25T21:11:51.739708+0000) 2024-04-25T21:12:18.471 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:18.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:36.440818+0000 front 2024-04-25T21:11:36.440838+0000 (oldest deadline 2024-04-25T21:12:01.740809+0000) 2024-04-25T21:12:18.472 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:18.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:46.242973+0000 front 2024-04-25T21:11:46.242998+0000 (oldest deadline 2024-04-25T21:12:11.542885+0000) 2024-04-25T21:12:18.472 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8[50968]: 2024-04-25T21:12:18.087+0000 7fc51ab1c700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:52.043960+0000 front 2024-04-25T21:11:52.044018+0000 (oldest deadline 2024-04-25T21:12:17.344159+0000) 2024-04-25T21:12:18.970 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:18 smithi186 podman[69697]: 2024-04-25 21:12:18.740116402 +0000 UTC m=+5.541268293 container died 71c931e8d29856e542d541556c285ec4780adbe940f25dd9f10c6eef5d3db6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:12:18.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:18.495+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:18.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:18.495+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:18.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:18.495+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:18.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:18.495+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:18.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:18.495+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:18.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:18 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:18.495+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:19.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:18.649+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:19.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:18.649+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:19.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:18.649+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:19.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:18.649+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:19.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:18.649+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:19.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:18 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:18.649+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:19.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:18.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:19.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:18.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:19.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:18.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:19.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:18.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:19.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:18.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:19.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:18 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:18.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:19.361 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:19 smithi186 podman[69697]: 2024-04-25 21:12:19.111021964 +0000 UTC m=+5.912173844 container cleanup 71c931e8d29856e542d541556c285ec4780adbe940f25dd9f10c6eef5d3db6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T21:12:19.361 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:19 smithi186 bash[69697]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8 2024-04-25T21:12:19.670 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:19 smithi186 podman[69714]: 2024-04-25 21:12:19.361991805 +0000 UTC m=+0.613439260 container remove 71c931e8d29856e542d541556c285ec4780adbe940f25dd9f10c6eef5d3db6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:12:19.670 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:19 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:19.465+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:19.670 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:19 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:19.465+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:19.670 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:19 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:19.465+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:19.670 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:19 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:19.465+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:19.671 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:19 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:19.465+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:19.671 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:19 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:19.465+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:19.970 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:19 smithi186 podman[69807]: 2024-04-25 21:12:19.671091961 +0000 UTC m=+0.114431401 container create ba5fb43883820a454e0800f41e8d0c0fa994ddb9e3366fc2e33fb905c6030831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T21:12:19.970 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:19 smithi186 podman[69807]: 2024-04-25 21:12:19.57449053 +0000 UTC m=+0.017829979 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:12:19.970 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:19 smithi186 podman[69807]: 2024-04-25 21:12:19.898517298 +0000 UTC m=+0.341856737 container init ba5fb43883820a454e0800f41e8d0c0fa994ddb9e3366fc2e33fb905c6030831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T21:12:19.970 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:19 smithi186 podman[69807]: 2024-04-25 21:12:19.901882639 +0000 UTC m=+0.345222088 container start ba5fb43883820a454e0800f41e8d0c0fa994ddb9e3366fc2e33fb905c6030831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T21:12:20.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:19 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:19.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:20.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:19 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:19.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:20.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:19 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:19.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:20.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:19 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:19.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:20.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:19 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:19.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:20.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:19 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:19.626+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:20.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:19 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:19.853+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:20.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:19 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:19.853+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:20.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:19 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:19.853+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:20.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:19 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:19.853+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:20.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:19 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:19.853+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:20.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:19 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:19.853+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:20.227 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:19 smithi186 podman[69807]: 2024-04-25 21:12:19.973339965 +0000 UTC m=+0.416679405 container attach ba5fb43883820a454e0800f41e8d0c0fa994ddb9e3366fc2e33fb905c6030831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T21:12:20.227 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 25 21:12:20 smithi186 podman[69807]: 2024-04-25 21:12:20.201172781 +0000 UTC m=+0.644512221 container died ba5fb43883820a454e0800f41e8d0c0fa994ddb9e3366fc2e33fb905c6030831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T21:12:20.480 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:20 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:20.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:20.480 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:20 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:20.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:20.480 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:20 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:20.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:20.480 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:20 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:20.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:20.481 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:20 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:20.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:20.481 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:20 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:20.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:20.862 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T21:12:20.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:20.903 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T21:12:20.903 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T21:12:20.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.9 2024-04-25T21:12:21.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:20 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:20.577+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:21.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:20 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:20.577+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:21.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:20 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:20.577+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:21.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:20 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:20.577+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:21.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:20 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:20.577+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:21.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:20 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:20.577+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:21.109 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:20 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:20.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:21.109 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:20 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:20.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:21.109 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:20 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:20.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:21.109 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:20 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:20.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:21.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:20 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:20.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:21.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:20 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:20.831+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:21.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:20 smithi047 systemd[1]: Stopping Ceph osd.9 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:21.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.354+0000 7fdf54c3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:21.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.354+0000 7fdf54c3e700 -1 osd.9 103 *** Got signal Terminated *** 2024-04-25T21:12:21.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.354+0000 7fdf54c3e700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:12:21.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:21.418+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:21.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:21.418+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:21.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:21.418+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:21.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:21.418+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:21.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:21.418+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:21.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:21.418+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:21.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:21 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:21.530+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:21.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:21 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:21.530+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:21.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:21 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:21.530+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:21.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:21 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:21.530+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:21.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:21 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:21.530+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:21.782 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:21 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:21.530+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:22.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.848+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:22.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.848+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:22.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.848+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:22.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.848+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:22.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.848+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:22.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:21 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:21.848+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:22.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:22 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:22.413+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:22.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:22 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:22.413+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:22.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:22 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:22.413+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:22.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:22 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:22.413+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:22.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:22 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:22.413+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:22.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:22 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:22.413+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:23.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:22 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:22.534+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:23.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:22 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:22.534+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:23.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:22 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:22.534+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:23.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:22 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:22.534+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:23.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:22 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:22.534+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:23.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:22 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:22.534+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:23.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:22 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:22.888+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:23.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:22 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:22.888+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:23.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:22 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:22.888+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:23.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:22 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:22.888+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:23.360 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:22 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:22.888+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:23.361 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:22 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:22.888+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:23.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:23 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:23.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:23.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:23 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:23.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:23.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:23 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:23.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:23.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:23 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:23.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:23.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:23 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:23.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:23.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:23 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:23.455+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:24.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:23 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:23.580+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:24.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:23 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:23.580+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:24.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:23 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:23.580+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:24.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:23 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:23.580+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:24.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:23 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:23.580+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:24.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:23 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:23.580+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:24.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:23.845+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:24.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:23.845+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:24.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:23.845+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:24.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:23.845+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:24.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:23.845+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:24.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:23 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:23.845+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:24.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:24 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:24.497+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:24.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:24 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:24.497+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:24.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:24 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:24.497+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:24.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:24 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:24.497+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:24.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:24 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:24.497+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:24.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:24 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:24.497+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:25.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:24 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:24.608+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:25.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:24 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:24.608+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:25.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:24 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:24.608+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:25.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:24 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:24.608+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:25.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:24 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:24.608+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:25.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:24 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:24.608+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:25.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:24 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:24.608+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:25.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:24 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:24.807+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:25.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:24 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:24.807+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:25.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:24 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:24.807+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:25.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:24 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:24.807+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:25.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:24 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:24.807+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:25.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:24 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:24.807+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:25.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:25 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:25.456+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:25.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:25 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:25.456+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:25.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:25 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:25.456+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:25.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:25 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:25.456+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:25.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:25 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:25.456+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:25.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:25 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:25.456+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:26.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:25 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:25.646+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:26.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:25 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:25.646+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:26.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:25 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:25.646+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:26.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:25 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:25.646+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:26.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:25 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:25.646+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:26.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:25 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:25.646+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:26.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:25 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:25.646+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:26.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:25 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:25.824+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:11.248672+0000 front 2024-04-25T21:11:11.248785+0000 (oldest deadline 2024-04-25T21:11:41.249550+0000) 2024-04-25T21:12:26.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:25 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:25.824+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:22.351034+0000 front 2024-04-25T21:11:22.351066+0000 (oldest deadline 2024-04-25T21:11:48.751264+0000) 2024-04-25T21:12:26.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:25 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:25.824+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:30.452677+0000 front 2024-04-25T21:11:30.452647+0000 (oldest deadline 2024-04-25T21:11:52.752367+0000) 2024-04-25T21:12:26.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:25 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:25.824+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:37.453225+0000 front 2024-04-25T21:11:37.453250+0000 (oldest deadline 2024-04-25T21:11:59.753411+0000) 2024-04-25T21:12:26.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:25 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:25.824+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:44.454561+0000 front 2024-04-25T21:11:44.454602+0000 (oldest deadline 2024-04-25T21:12:07.855129+0000) 2024-04-25T21:12:26.111 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:25 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9[61915]: 2024-04-25T21:12:25.824+0000 7fdf51232700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.656352+0000 front 2024-04-25T21:11:53.656434+0000 (oldest deadline 2024-04-25T21:12:15.356407+0000) 2024-04-25T21:12:26.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:26.459+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:26.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:26.459+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:26.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:26.459+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:26.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:26.459+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:26.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:26.459+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:26.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:26 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:26.459+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:27.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:26 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:26.609+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:27.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:26 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:26.609+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:27.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:26 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:26.609+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:27.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:26 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:26.609+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:27.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:26 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:26.609+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:27.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:26 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:26.609+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:27.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:26 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:26.609+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:27.110 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:26 smithi047 podman[114598]: 2024-04-25 21:12:26.84533746 +0000 UTC m=+5.711339763 container died e83588f1eae4de12df22a5551849b62a8f1f460be254da1a809a1159a21a84bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T21:12:27.513 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:27 smithi047 podman[114598]: 2024-04-25 21:12:27.204442478 +0000 UTC m=+6.070444779 container cleanup e83588f1eae4de12df22a5551849b62a8f1f460be254da1a809a1159a21a84bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:12:27.513 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:27 smithi047 bash[114598]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9 2024-04-25T21:12:27.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:27 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:27.450+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:27.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:27 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:27.450+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:27.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:27 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:27.450+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:27.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:27 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:27.450+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:27.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:27 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:27.450+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:27.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:27 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:27.450+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:27.860 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:27 smithi047 podman[114617]: 2024-04-25 21:12:27.513723433 +0000 UTC m=+0.663987120 container remove e83588f1eae4de12df22a5551849b62a8f1f460be254da1a809a1159a21a84bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:12:28.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:27 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:27.597+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:28.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:27 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:27.597+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:28.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:27 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:27.597+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:28.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:27 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:27.597+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:28.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:27 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:27.597+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:28.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:27 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:27.597+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:28.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:27 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:27.597+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:28.132 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:27 smithi047 podman[114708]: 2024-04-25 21:12:27.872911977 +0000 UTC m=+0.104251772 container create 27ea7dd995ad60ec2ae46827c885835c47d3ca2f6db3c0ee9abecb4cf6361859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T21:12:28.133 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:27 smithi047 podman[114708]: 2024-04-25 21:12:27.787428602 +0000 UTC m=+0.018768374 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:12:28.133 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:28 smithi047 podman[114708]: 2024-04-25 21:12:28.05865165 +0000 UTC m=+0.289991403 container init 27ea7dd995ad60ec2ae46827c885835c47d3ca2f6db3c0ee9abecb4cf6361859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T21:12:28.133 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:28 smithi047 podman[114708]: 2024-04-25 21:12:28.061753503 +0000 UTC m=+0.293093258 container start 27ea7dd995ad60ec2ae46827c885835c47d3ca2f6db3c0ee9abecb4cf6361859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T21:12:28.133 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:28 smithi047 podman[114708]: 2024-04-25 21:12:28.133452143 +0000 UTC m=+0.364791912 container attach 27ea7dd995ad60ec2ae46827c885835c47d3ca2f6db3c0ee9abecb4cf6361859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T21:12:28.610 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 25 21:12:28 smithi047 podman[114708]: 2024-04-25 21:12:28.364269405 +0000 UTC m=+0.595609148 container died 27ea7dd995ad60ec2ae46827c885835c47d3ca2f6db3c0ee9abecb4cf6361859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T21:12:28.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:28 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:28.437+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:28.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:28 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:28.437+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:28.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:28 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:28.437+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:28.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:28 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:28.437+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:28.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:28 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:28.437+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:28.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:28 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:28.437+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:28.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T21:12:29.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:29.002 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T21:12:29.002 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T21:12:29.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.10 2024-04-25T21:12:29.028 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:28 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:28.594+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:29.028 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:28 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:28.594+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:29.028 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:28 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:28.594+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:29.028 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:28 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:28.594+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:29.028 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:28 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:28.594+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:29.028 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:28 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:28.594+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:29.028 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:28 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:28.594+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:29.281 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 systemd[1]: Stopping Ceph osd.10 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:29.592 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.292+0000 7f6b4be57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:29.593 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.292+0000 7f6b4be57700 -1 osd.10 103 *** Got signal Terminated *** 2024-04-25T21:12:29.593 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.292+0000 7f6b4be57700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:12:29.593 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.592+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:29.593 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.592+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:29.593 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.592+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:29.593 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.592+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:29.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:29 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:29.460+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:29.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:29 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:29.460+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:29.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:29 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:29.460+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:29.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:29 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:29.460+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:29.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:29 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:29.460+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:29.720 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:29 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:29.460+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:30.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.592+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:30.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.592+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:30.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:29 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:29.592+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:30.917 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:30 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:30.477+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:30.917 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:30 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:30.477+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:30.917 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:30 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:30.477+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:30.917 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:30 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:30.477+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:30.918 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:30 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:30.477+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:30.918 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:30 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:30.477+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:31.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:30 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:30.578+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:31.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:30 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:30.578+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:31.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:30 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:30.578+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:31.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:30 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:30.578+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:31.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:30 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:30.578+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:31.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:30 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:30.578+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:31.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:30 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:30.578+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:31.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:31 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:31.500+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:31.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:31 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:31.500+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:31.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:31 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:31.500+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:31.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:31 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:31.500+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:31.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:31 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:31.500+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:31.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:31 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:31.500+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:31.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:31 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:31.500+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:32.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:31 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:31.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:32.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:31 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:31.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:32.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:31 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:31.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:32.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:31 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:31.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:32.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:31 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:31.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:32.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:31 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:31.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:32.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:31 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:31.629+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:32.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:32.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:32.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:32.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:32.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:32.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:32.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:32.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:32.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:32.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:32.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:32.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:32.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:32 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:32.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:33.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:32 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:32.612+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:33.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:32 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:32.612+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:33.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:32 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:32.612+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:33.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:32 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:32.612+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:33.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:32 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:32.612+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:33.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:32 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:32.612+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:33.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:32 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:32.612+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:33.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:33 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:33.567+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:33.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:33 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:33.567+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:33.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:33 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:33.567+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:33.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:33 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:33.567+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:33.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:33 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:33.567+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:33.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:33 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:33.567+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:33.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:33 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:33.567+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:34.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:33 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:33.653+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.027008+0000 front 2024-04-25T21:11:14.026992+0000 (oldest deadline 2024-04-25T21:11:37.527130+0000) 2024-04-25T21:12:34.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:33 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:33.653+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.628063+0000 front 2024-04-25T21:11:21.628158+0000 (oldest deadline 2024-04-25T21:11:47.528374+0000) 2024-04-25T21:12:34.031 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:33 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:33.653+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.829466+0000 front 2024-04-25T21:11:29.829399+0000 (oldest deadline 2024-04-25T21:11:52.129508+0000) 2024-04-25T21:12:34.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:33 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:33.653+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.431940+0000 front 2024-04-25T21:11:38.431885+0000 (oldest deadline 2024-04-25T21:12:02.531741+0000) 2024-04-25T21:12:34.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:33 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:33.653+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.532079+0000 front 2024-04-25T21:11:42.532096+0000 (oldest deadline 2024-04-25T21:12:07.832087+0000) 2024-04-25T21:12:34.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:33 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:33.653+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:54.233444+0000 front 2024-04-25T21:11:54.233333+0000 (oldest deadline 2024-04-25T21:12:15.833700+0000) 2024-04-25T21:12:34.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:33 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10[57165]: 2024-04-25T21:12:33.653+0000 7f6b44443700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-25T21:11:58.134836+0000 front 2024-04-25T21:11:58.134758+0000 (oldest deadline 2024-04-25T21:12:24.034555+0000) 2024-04-25T21:12:34.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:34 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:34.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:34.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:34 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:34.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:34.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:34 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:34.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:34.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:34 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:34.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:34.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:34 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:34.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:34.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:34 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:34.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:34.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:34 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:34.522+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:35.032 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:34 smithi092 podman[69923]: 2024-04-25 21:12:34.747189158 +0000 UTC m=+5.551114149 container died 57ad398166e5df205e208bbbba189eb44155690fe785b77fe3d55ff65a5e733b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T21:12:35.531 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:35 smithi092 podman[69923]: 2024-04-25 21:12:35.067303862 +0000 UTC m=+5.871228851 container cleanup 57ad398166e5df205e208bbbba189eb44155690fe785b77fe3d55ff65a5e733b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T21:12:35.532 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:35 smithi092 bash[69923]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10 2024-04-25T21:12:35.532 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:35 smithi092 podman[69941]: 2024-04-25 21:12:35.260120455 +0000 UTC m=+0.511812915 container remove 57ad398166e5df205e208bbbba189eb44155690fe785b77fe3d55ff65a5e733b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T21:12:35.962 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:35 smithi092 podman[70032]: 2024-04-25 21:12:35.562283126 +0000 UTC m=+0.017621701 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:12:35.962 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:35 smithi092 podman[70032]: 2024-04-25 21:12:35.669185153 +0000 UTC m=+0.124523721 container create 7a8dc80cd75f010e0f9bc300b0ca920b2006f2928c21a004f34eebd1cdc89c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:12:35.963 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:35 smithi092 podman[70032]: 2024-04-25 21:12:35.896872055 +0000 UTC m=+0.352210622 container init 7a8dc80cd75f010e0f9bc300b0ca920b2006f2928c21a004f34eebd1cdc89c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T21:12:35.963 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:35 smithi092 podman[70032]: 2024-04-25 21:12:35.901015828 +0000 UTC m=+0.356354402 container start 7a8dc80cd75f010e0f9bc300b0ca920b2006f2928c21a004f34eebd1cdc89c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T21:12:35.963 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:35 smithi092 podman[70032]: 2024-04-25 21:12:35.963320418 +0000 UTC m=+0.418659013 container attach 7a8dc80cd75f010e0f9bc300b0ca920b2006f2928c21a004f34eebd1cdc89c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:12:35.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:35 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:35.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:35.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:35 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:35.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:35.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:35 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:35.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:35.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:35 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:35.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:35.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:35 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:35.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:35.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:35 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:35.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:35.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:35 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:35.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:36.281 INFO:journalctl@ceph.osd.10.smithi092.stdout:Apr 25 21:12:36 smithi092 podman[70032]: 2024-04-25 21:12:36.195268463 +0000 UTC m=+0.650607020 container died 7a8dc80cd75f010e0f9bc300b0ca920b2006f2928c21a004f34eebd1cdc89c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T21:12:36.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T21:12:36.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:36.836 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T21:12:36.836 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T21:12:36.836 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.11 2024-04-25T21:12:36.862 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:36 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:36.510+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:36.862 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:36 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:36.510+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:36.862 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:36 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:36.510+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:36.862 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:36 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:36.510+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:36.863 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:36 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:36.510+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:36.863 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:36 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:36.510+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:36.863 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:36 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:36.510+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:37.129 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:36 smithi186 systemd[1]: Stopping Ceph osd.11 for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:37.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.129+0000 7f5f94e36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:37.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.129+0000 7f5f94e36700 -1 osd.11 103 *** Got signal Terminated *** 2024-04-25T21:12:37.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.129+0000 7f5f94e36700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:12:37.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.555+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:37.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.555+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:37.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.555+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:37.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.555+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:37.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.555+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:37.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.555+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:37.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:37 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:37.555+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:38.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:38.566+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:38.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:38.566+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:38.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:38.566+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:38.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:38.566+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:38.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:38.566+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:38.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:38.566+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:38.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:38 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:38.566+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:39.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:39.572+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:39.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:39.572+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:39.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:39.572+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:39.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:39.572+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:39.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:39.572+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:39.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:39.572+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:39.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:39 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:39.572+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:40.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:40.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:40.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:40.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:40.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:40.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:40.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:40.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:40.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:40.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:40.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:40.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:40.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:40 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:40.530+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:41.544+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-25T21:11:14.542952+0000 front 2024-04-25T21:11:14.542998+0000 (oldest deadline 2024-04-25T21:11:38.043082+0000) 2024-04-25T21:12:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:41.544+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-04-25T21:11:21.544229+0000 front 2024-04-25T21:11:21.544205+0000 (oldest deadline 2024-04-25T21:11:46.244194+0000) 2024-04-25T21:12:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:41.544+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-25T21:11:29.144957+0000 front 2024-04-25T21:11:29.145002+0000 (oldest deadline 2024-04-25T21:11:55.045183+0000) 2024-04-25T21:12:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:41.544+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-25T21:11:38.546192+0000 front 2024-04-25T21:11:38.546139+0000 (oldest deadline 2024-04-25T21:12:02.646261+0000) 2024-04-25T21:12:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:41.544+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-04-25T21:11:42.646969+0000 front 2024-04-25T21:11:42.646876+0000 (oldest deadline 2024-04-25T21:12:08.546865+0000) 2024-04-25T21:12:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:41.544+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-25T21:11:53.847648+0000 front 2024-04-25T21:11:53.847680+0000 (oldest deadline 2024-04-25T21:12:16.147626+0000) 2024-04-25T21:12:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:41.544+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-04-25T21:12:05.549280+0000 front 2024-04-25T21:12:05.549315+0000 (oldest deadline 2024-04-25T21:12:30.849357+0000) 2024-04-25T21:12:41.970 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:41 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11[57558]: 2024-04-25T21:12:41.544+0000 7f5f9142a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-04-25T21:12:16.650846+0000 front 2024-04-25T21:12:16.650819+0000 (oldest deadline 2024-04-25T21:12:40.750866+0000) 2024-04-25T21:12:42.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:42 smithi186 podman[69925]: 2024-04-25 21:12:42.585104826 +0000 UTC m=+5.550911115 container died 4265d58000db35542376086c08bfcd36ad1dc1aea743190b94a4302bef5c11e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:12:43.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:42 smithi186 podman[69925]: 2024-04-25 21:12:42.928708526 +0000 UTC m=+5.894514819 container cleanup 4265d58000db35542376086c08bfcd36ad1dc1aea743190b94a4302bef5c11e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T21:12:43.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:42 smithi186 bash[69925]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11 2024-04-25T21:12:43.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:43 smithi186 podman[69942]: 2024-04-25 21:12:43.088108465 +0000 UTC m=+0.504248970 container remove 4265d58000db35542376086c08bfcd36ad1dc1aea743190b94a4302bef5c11e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T21:12:43.782 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:43 smithi186 podman[70036]: 2024-04-25 21:12:43.399381303 +0000 UTC m=+0.017684604 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:12:43.782 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:43 smithi186 podman[70036]: 2024-04-25 21:12:43.497116058 +0000 UTC m=+0.115419334 container create 529cd49a4ccf18470c497bdeb5fc4435acfd7254a6e8fc4806f87e1a20054013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T21:12:43.782 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:43 smithi186 podman[70036]: 2024-04-25 21:12:43.717035389 +0000 UTC m=+0.335338685 container init 529cd49a4ccf18470c497bdeb5fc4435acfd7254a6e8fc4806f87e1a20054013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:12:43.782 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:43 smithi186 podman[70036]: 2024-04-25 21:12:43.720240608 +0000 UTC m=+0.338543886 container start 529cd49a4ccf18470c497bdeb5fc4435acfd7254a6e8fc4806f87e1a20054013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T21:12:43.782 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:43 smithi186 podman[70036]: 2024-04-25 21:12:43.783048435 +0000 UTC m=+0.401351729 container attach 529cd49a4ccf18470c497bdeb5fc4435acfd7254a6e8fc4806f87e1a20054013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T21:12:44.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 25 21:12:44 smithi186 podman[70036]: 2024-04-25 21:12:44.01021671 +0000 UTC m=+0.628519989 container died 529cd49a4ccf18470c497bdeb5fc4435acfd7254a6e8fc4806f87e1a20054013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T21:12:44.559 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T21:12:44.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:44.636 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T21:12:44.636 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T21:12:44.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.a 2024-04-25T21:12:44.938 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 21:12:44 smithi047 systemd[1]: Stopping Ceph mds.a for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:45.360 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 21:12:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a[68916]: 2024-04-25T21:12:44.938+0000 7f98dc845700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:45.360 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 21:12:44 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a[68916]: 2024-04-25T21:12:44.938+0000 7f98dc845700 -1 mds.a *** got signal Terminated *** 2024-04-25T21:12:46.879 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 21:12:46 smithi047 podman[114815]: 2024-04-25 21:12:46.611842513 +0000 UTC m=+1.770876290 container died 6b2424b7f2039a8ea6cca8dd2cdc5363acf773ea872db349d213be5cad680b36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-25T21:12:47.232 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 21:12:46 smithi047 podman[114815]: 2024-04-25 21:12:46.971125543 +0000 UTC m=+2.130159320 container cleanup 6b2424b7f2039a8ea6cca8dd2cdc5363acf773ea872db349d213be5cad680b36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:12:47.232 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 25 21:12:46 smithi047 bash[114815]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-a 2024-04-25T21:12:47.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T21:12:47.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:47.450 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T21:12:47.451 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T21:12:47.451 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.d 2024-04-25T21:12:47.776 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:47 smithi047 systemd[1]: Stopping Ceph mds.d for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:48.110 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:47 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d[67683]: 2024-04-25T21:12:47.775+0000 7f1c74b09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:48.110 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:47 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d[67683]: 2024-04-25T21:12:47.775+0000 7f1c74b09700 -1 mds.d *** got signal Terminated *** 2024-04-25T21:12:50.243 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:49 smithi047 podman[114946]: 2024-04-25 21:12:49.923318206 +0000 UTC m=+2.244487279 container died 279f0681eb37e63964968a7887cacd97b5db9fd25967dddedd0050f4cf8a4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T21:12:50.560 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:50 smithi047 podman[114946]: 2024-04-25 21:12:50.343755611 +0000 UTC m=+2.664924671 container cleanup 279f0681eb37e63964968a7887cacd97b5db9fd25967dddedd0050f4cf8a4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T21:12:50.560 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:50 smithi047 bash[114946]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d 2024-04-25T21:12:50.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T21:12:50.813 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:50 smithi047 podman[114960]: 2024-04-25 21:12:50.56135558 +0000 UTC m=+0.628353678 container remove 279f0681eb37e63964968a7887cacd97b5db9fd25967dddedd0050f4cf8a4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T21:12:50.813 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:50 smithi047 systemd[1]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-25T21:12:50.813 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:50 smithi047 systemd[1]: Stopped Ceph mds.d for 0ca8de06-0345-11ef-bc93-c7b262605968. 2024-04-25T21:12:50.813 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 25 21:12:50 smithi047 systemd[1]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.d.service: Consumed 1.125s CPU time. 2024-04-25T21:12:50.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:50.825 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T21:12:50.825 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T21:12:50.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.g 2024-04-25T21:12:51.110 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 21:12:50 smithi047 systemd[1]: Stopping Ceph mds.g for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:51.610 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 21:12:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g[68495]: 2024-04-25T21:12:51.138+0000 7fbc12083700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:51.610 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 21:12:51 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g[68495]: 2024-04-25T21:12:51.138+0000 7fbc12083700 -1 mds.g *** got signal Terminated *** 2024-04-25T21:12:55.406 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 21:12:55 smithi047 podman[115069]: 2024-04-25 21:12:55.153788362 +0000 UTC m=+4.108774733 container died 2f3d7d4e366703580f5336921555d3f92bcdfdce477d6e94d7e7eddc60c0ab69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T21:12:55.673 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 21:12:55 smithi047 podman[115069]: 2024-04-25 21:12:55.48035241 +0000 UTC m=+4.435338790 container cleanup 2f3d7d4e366703580f5336921555d3f92bcdfdce477d6e94d7e7eddc60c0ab69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:12:55.673 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 25 21:12:55 smithi047 bash[115069]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-g 2024-04-25T21:12:55.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T21:12:55.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:55.932 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T21:12:55.932 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T21:12:55.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.j 2024-04-25T21:12:56.258 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 21:12:56 smithi047 systemd[1]: Stopping Ceph mds.j for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:56.610 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 21:12:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j[67239]: 2024-04-25T21:12:56.258+0000 7fd6ad81b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:56.610 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 21:12:56 smithi047 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j[67239]: 2024-04-25T21:12:56.258+0000 7fd6ad81b700 -1 mds.j *** got signal Terminated *** 2024-04-25T21:12:58.600 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 21:12:58 smithi047 podman[115191]: 2024-04-25 21:12:58.333833537 +0000 UTC m=+2.172529108 container died 594afccd3cb2e4339c31b53aeecc6c94b155a460441b3856386aeb2f305797cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T21:12:58.851 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 21:12:58 smithi047 podman[115191]: 2024-04-25 21:12:58.675799353 +0000 UTC m=+2.514494923 container cleanup 594afccd3cb2e4339c31b53aeecc6c94b155a460441b3856386aeb2f305797cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T21:12:58.852 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 21:12:58 smithi047 bash[115191]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j 2024-04-25T21:12:58.852 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 25 21:12:58 smithi047 podman[115205]: 2024-04-25 21:12:58.851657095 +0000 UTC m=+0.514597315 container remove 594afccd3cb2e4339c31b53aeecc6c94b155a460441b3856386aeb2f305797cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:12:59.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T21:12:59.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:12:59.145 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T21:12:59.145 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T21:12:59.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.b 2024-04-25T21:12:59.449 INFO:journalctl@ceph.mds.b.smithi092.stdout:Apr 25 21:12:59 smithi092 systemd[1]: Stopping Ceph mds.b for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:12:59.781 INFO:journalctl@ceph.mds.b.smithi092.stdout:Apr 25 21:12:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-b[62808]: 2024-04-25T21:12:59.448+0000 7fbd8ee09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:12:59.782 INFO:journalctl@ceph.mds.b.smithi092.stdout:Apr 25 21:12:59 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-b[62808]: 2024-04-25T21:12:59.448+0000 7fbd8ee09700 -1 mds.b *** got signal Terminated *** 2024-04-25T21:13:00.781 INFO:journalctl@ceph.mds.b.smithi092.stdout:Apr 25 21:13:00 smithi092 podman[70149]: 2024-04-25 21:13:00.451858736 +0000 UTC m=+1.109691358 container died 2e73444aa077032a0b9798c7086eafd718a5fe22c379db7df75058409934dc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T21:13:01.075 INFO:journalctl@ceph.mds.b.smithi092.stdout:Apr 25 21:13:00 smithi092 podman[70149]: 2024-04-25 21:13:00.883024703 +0000 UTC m=+1.540857333 container cleanup 2e73444aa077032a0b9798c7086eafd718a5fe22c379db7df75058409934dc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T21:13:01.075 INFO:journalctl@ceph.mds.b.smithi092.stdout:Apr 25 21:13:00 smithi092 bash[70149]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-b 2024-04-25T21:13:01.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T21:13:01.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:13:01.334 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T21:13:01.334 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T21:13:01.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.e 2024-04-25T21:13:01.644 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:13:01 smithi092 systemd[1]: Stopping Ceph mds.e for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:13:02.031 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:13:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-e[62181]: 2024-04-25T21:13:01.644+0000 7f5a59aa6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:13:02.032 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:13:01 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-e[62181]: 2024-04-25T21:13:01.644+0000 7f5a59aa6700 -1 mds.e *** got signal Terminated *** 2024-04-25T21:13:04.720 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:13:04 smithi092 podman[70270]: 2024-04-25 21:13:04.471216536 +0000 UTC m=+2.911948697 container died edd06b8f023629b5a3cdec4ac12617cbc263d815d94fb6c5a830dc680e44c6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T21:13:05.031 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:13:04 smithi092 podman[70270]: 2024-04-25 21:13:04.828079471 +0000 UTC m=+3.268811631 container cleanup edd06b8f023629b5a3cdec4ac12617cbc263d815d94fb6c5a830dc680e44c6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T21:13:05.031 INFO:journalctl@ceph.mds.e.smithi092.stdout:Apr 25 21:13:04 smithi092 bash[70270]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-e 2024-04-25T21:13:05.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T21:13:05.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:13:05.237 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T21:13:05.237 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T21:13:05.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.h 2024-04-25T21:13:05.531 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:13:05 smithi092 systemd[1]: Stopping Ceph mds.h for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:13:06.031 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:13:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-h[62485]: 2024-04-25T21:13:05.551+0000 7f85bb972700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:13:06.031 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:13:05 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-h[62485]: 2024-04-25T21:13:05.551+0000 7f85bb972700 -1 mds.h *** got signal Terminated *** 2024-04-25T21:13:08.750 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:13:08 smithi092 podman[70390]: 2024-04-25 21:13:08.46181916 +0000 UTC m=+3.008508233 container died c2973cdf983ee43ee700b4b7aa8d0c7d5285aaa8a01162cf4d266ea3ea55efcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T21:13:09.031 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:13:08 smithi092 podman[70390]: 2024-04-25 21:13:08.848841456 +0000 UTC m=+3.395530563 container cleanup c2973cdf983ee43ee700b4b7aa8d0c7d5285aaa8a01162cf4d266ea3ea55efcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T21:13:09.031 INFO:journalctl@ceph.mds.h.smithi092.stdout:Apr 25 21:13:08 smithi092 bash[70390]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-h 2024-04-25T21:13:09.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T21:13:09.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:13:09.269 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T21:13:09.269 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T21:13:09.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.k 2024-04-25T21:13:09.781 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 21:13:09 smithi092 systemd[1]: Stopping Ceph mds.k for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:13:09.782 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 21:13:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k[61873]: 2024-04-25T21:13:09.568+0000 7f92392d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:13:09.782 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 21:13:09 smithi092 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k[61873]: 2024-04-25T21:13:09.568+0000 7f92392d4700 -1 mds.k *** got signal Terminated *** 2024-04-25T21:13:13.835 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 21:13:13 smithi092 podman[70523]: 2024-04-25 21:13:13.582446487 +0000 UTC m=+4.096971065 container died 76869c1f7439d2ead7479878893fe7c6e25ebd1d290faee6c3eddbb8dd0be165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:13:14.210 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 21:13:13 smithi092 podman[70523]: 2024-04-25 21:13:13.927050703 +0000 UTC m=+4.441575278 container cleanup 76869c1f7439d2ead7479878893fe7c6e25ebd1d290faee6c3eddbb8dd0be165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T21:13:14.210 INFO:journalctl@ceph.mds.k.smithi092.stdout:Apr 25 21:13:13 smithi092 bash[70523]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-k 2024-04-25T21:13:14.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T21:13:14.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:13:14.385 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T21:13:14.385 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T21:13:14.385 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.c 2024-04-25T21:13:14.694 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:13:14 smithi186 systemd[1]: Stopping Ceph mds.c for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:13:14.952 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:13:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c[62450]: 2024-04-25T21:13:14.693+0000 7f40d3766700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:13:14.952 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:13:14 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c[62450]: 2024-04-25T21:13:14.693+0000 7f40d3766700 -1 mds.c *** got signal Terminated *** 2024-04-25T21:13:16.719 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:13:16 smithi186 podman[70139]: 2024-04-25 21:13:16.457637207 +0000 UTC m=+1.878773123 container died 3e8dbccef69829b0ad93dc60769edab1e70ca9569c627969b9b92fcbb4792fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T21:13:17.161 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:13:16 smithi186 podman[70139]: 2024-04-25 21:13:16.902279326 +0000 UTC m=+2.323415252 container cleanup 3e8dbccef69829b0ad93dc60769edab1e70ca9569c627969b9b92fcbb4792fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T21:13:17.161 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 25 21:13:16 smithi186 bash[70139]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-c 2024-04-25T21:13:17.312 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T21:13:17.347 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:13:17.348 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T21:13:17.348 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T21:13:17.348 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.f 2024-04-25T21:13:17.719 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:13:17 smithi186 systemd[1]: Stopping Ceph mds.f for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:13:17.720 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:13:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f[62141]: 2024-04-25T21:13:17.663+0000 7f0b6756b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:13:17.720 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:13:17 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f[62141]: 2024-04-25T21:13:17.663+0000 7f0b6756b700 -1 mds.f *** got signal Terminated *** 2024-04-25T21:13:20.719 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:13:20 smithi186 podman[70270]: 2024-04-25 21:13:20.460823296 +0000 UTC m=+2.897903586 container died dd9898b1495e4f8d9de885bcffdca69bc8c5c34486d7b82ee2181c7f746e7426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:13:21.114 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:13:20 smithi186 podman[70270]: 2024-04-25 21:13:20.805717403 +0000 UTC m=+3.242797690 container cleanup dd9898b1495e4f8d9de885bcffdca69bc8c5c34486d7b82ee2181c7f746e7426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T21:13:21.114 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 25 21:13:20 smithi186 bash[70270]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-f 2024-04-25T21:13:21.232 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T21:13:21.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:13:21.264 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T21:13:21.264 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T21:13:21.264 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.i 2024-04-25T21:13:21.469 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:13:21 smithi186 systemd[1]: Stopping Ceph mds.i for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:13:21.969 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:13:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i[62769]: 2024-04-25T21:13:21.583+0000 7f3f3ce49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:13:21.970 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:13:21 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i[62769]: 2024-04-25T21:13:21.583+0000 7f3f3ce49700 -1 mds.i *** got signal Terminated *** 2024-04-25T21:13:24.720 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:13:24 smithi186 podman[70389]: 2024-04-25 21:13:24.451769885 +0000 UTC m=+2.970773826 container died 655565dc0158c134d7909087424ec496591f972b6646b3dcb2b54ccc85459636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-25T21:13:25.061 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:13:24 smithi186 podman[70389]: 2024-04-25 21:13:24.83542844 +0000 UTC m=+3.354432390 container cleanup 655565dc0158c134d7909087424ec496591f972b6646b3dcb2b54ccc85459636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T21:13:25.061 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 25 21:13:24 smithi186 bash[70389]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-i 2024-04-25T21:13:25.189 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T21:13:25.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:13:25.219 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T21:13:25.220 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T21:13:25.220 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.l 2024-04-25T21:13:25.381 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 21:13:25 smithi186 systemd[1]: Stopping Ceph mds.l for 0ca8de06-0345-11ef-bc93-c7b262605968... 2024-04-25T21:13:25.719 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 21:13:25 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l[61828]: 2024-04-25T21:13:25.539+0000 7f35a39f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:13:25.719 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 21:13:25 smithi186 ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l[61828]: 2024-04-25T21:13:25.539+0000 7f35a39f3700 -1 mds.l *** got signal Terminated *** 2024-04-25T21:13:27.598 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 21:13:27 smithi186 podman[70509]: 2024-04-25 21:13:27.275742032 +0000 UTC m=+1.833360563 container died de88cee1ac37004e0da2563989d47b5d0f6a9b6e3f37d3e93df1d6f3df390143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T21:13:27.906 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 21:13:27 smithi186 podman[70509]: 2024-04-25 21:13:27.681515627 +0000 UTC m=+2.239134163 container cleanup de88cee1ac37004e0da2563989d47b5d0f6a9b6e3f37d3e93df1d6f3df390143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T21:13:27.906 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 25 21:13:27 smithi186 bash[70509]: ceph-0ca8de06-0345-11ef-bc93-c7b262605968-mds-l 2024-04-25T21:13:28.041 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ca8de06-0345-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T21:13:28.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:13:28.071 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T21:13:28.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:13:28.271 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T21:13:30.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:13:31.084 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T21:13:33.372 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:13:33.562 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T21:13:35.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:13:35.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:13:35.972 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:13:35.998 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T21:13:36.000 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi047/crash 2024-04-25T21:13:36.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:13:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:13:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:13:36.049 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi092/crash 2024-04-25T21:13:36.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:13:36.087 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:13:36.088 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:13:36.089 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi186/crash 2024-04-25T21:13:36.090 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:13:36.129 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:13:36.129 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:13:36.130 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T21:13:36.130 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T21:13:36.168 INFO:tasks.cephadm:Compressing logs... 2024-04-25T21:13:36.168 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:13:36.211 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:13:36.214 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:13:36.237 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:13:36.237 INFO:teuthology.orchestra.run.smithi186.stderr:: No such file or directory 2024-04-25T21:13:36.237 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T21:13:36.237 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T21:13:36.237 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:13:36.237 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T21:13:36.244 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T21:13:36.244 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T21:13:36.244 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.5.log: /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T21:13:36.244 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T21:13:36.245 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.z.log: /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.c.log: 87.4% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:13:36.245 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:13:36.245 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:13:36.253 INFO:teuthology.orchestra.run.smithi186.stderr: 91.9% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T21:13:36.254 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T21:13:36.254 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T21:13:36.254 INFO:teuthology.orchestra.run.smithi186.stderr: 83.8% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:13:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:13:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:: No such file or directory 2024-04-25T21:13:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:13:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T21:13:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T21:13:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T21:13:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T21:13:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- 90.3% /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.2% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr: 94.5% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr: 94.5% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T21:13:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.4%gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:13:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.h.log: /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.10.log: 90.5% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:13:36.260 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T21:13:36.260 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:13:36.260 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:13:36.260 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T21:13:36.260 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T21:13:36.260 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:13:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T21:13:36.261 INFO:teuthology.orchestra.run.smithi047.stderr: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:13:36.267 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T21:13:36.269 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log.gzgzip -5 --verbose -- /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T21:13:36.269 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-25T21:13:36.275 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:13:36.278 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:13:36.293 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T21:13:36.293 INFO:teuthology.orchestra.run.smithi186.stderr: 91.0% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T21:13:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T21:13:36.296 INFO:teuthology.orchestra.run.smithi186.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:13:36.300 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:13:36.316 INFO:teuthology.orchestra.run.smithi186.stderr: 94.1% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T21:13:36.433 INFO:teuthology.orchestra.run.smithi186.stderr: 94.4% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T21:13:36.497 INFO:teuthology.orchestra.run.smithi047.stderr: 90.0% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T21:13:36.560 INFO:teuthology.orchestra.run.smithi186.stderr: 92.0% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T21:13:36.624 INFO:teuthology.orchestra.run.smithi092.stderr: 93.9% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T21:13:36.736 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T21:13:37.163 INFO:teuthology.orchestra.run.smithi047.stderr: 90.5% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T21:13:38.228 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T21:13:38.418 INFO:teuthology.orchestra.run.smithi186.stderr: 96.2% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T21:13:38.900 INFO:teuthology.orchestra.run.smithi186.stderr: 94.2% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T21:13:39.388 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T21:13:39.465 INFO:teuthology.orchestra.run.smithi092.stderr: 94.1% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T21:13:39.958 INFO:teuthology.orchestra.run.smithi186.stderr: 93.8% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T21:13:40.195 INFO:teuthology.orchestra.run.smithi047.stderr: 94.1% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T21:13:40.308 INFO:teuthology.orchestra.run.smithi047.stderr: 93.9% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T21:13:40.485 INFO:teuthology.orchestra.run.smithi047.stderr: 93.9% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T21:13:40.551 INFO:teuthology.orchestra.run.smithi186.stderr: 94.2% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T21:13:41.391 INFO:teuthology.orchestra.run.smithi047.stderr: 93.8% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T21:13:41.392 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-25T21:13:41.393 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m5.167s 2024-04-25T21:13:41.393 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m18.044s 2024-04-25T21:13:41.393 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.599s 2024-04-25T21:13:41.470 INFO:teuthology.orchestra.run.smithi186.stderr: 94.1% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T21:13:41.472 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-25T21:13:41.472 INFO:teuthology.orchestra.run.smithi186.stderr:real 0m5.244s 2024-04-25T21:13:41.472 INFO:teuthology.orchestra.run.smithi186.stderr:user 0m18.029s 2024-04-25T21:13:41.472 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m0.648s 2024-04-25T21:13:41.963 INFO:teuthology.orchestra.run.smithi092.stderr: 93.9% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T21:13:42.966 INFO:teuthology.orchestra.run.smithi092.stderr: 94.1% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T21:14:05.684 INFO:teuthology.orchestra.run.smithi092.stderr: 93.3% -- replaced with /var/log/ceph/0ca8de06-0345-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T21:14:05.686 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-25T21:14:05.686 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m29.459s 2024-04-25T21:14:05.687 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m49.052s 2024-04-25T21:14:05.687 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m1.606s 2024-04-25T21:14:05.687 INFO:tasks.cephadm:Archiving logs... 2024-04-25T21:14:05.687 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi047/log 2024-04-25T21:14:05.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:14:08.091 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi092/log 2024-04-25T21:14:08.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:14:14.661 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi186/log 2024-04-25T21:14:14.662 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:14:16.909 INFO:tasks.cephadm:Removing cluster... 2024-04-25T21:14:16.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 --force 2024-04-25T21:14:17.103 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T21:14:17.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 --force 2024-04-25T21:14:17.566 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T21:14:17.881 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ca8de06-0345-11ef-bc93-c7b262605968 --force 2024-04-25T21:14:18.074 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 0ca8de06-0345-11ef-bc93-c7b262605968 2024-04-25T21:14:18.404 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T21:14:18.404 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:14:18.423 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:14:18.440 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:14:18.455 INFO:tasks.cephadm:Teardown complete 2024-04-25T21:14:18.455 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T21:14:18.467 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T21:14:18.467 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:14:18.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:14:18.483 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:14:18.579 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T21:14:18.591 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T21:14:18.592 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:14:18.595 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:14:18.597 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:14:18.609 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:14:18.609 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:14:18.610 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:14:18.613 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:14:18.614 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-04-25T21:14:18.614 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +472us[ +478us] +/- 66ms 2024-04-25T21:14:18.614 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -1181us[-1181us] +/- 72ms 2024-04-25T21:14:18.614 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +722us[ +728us] +/- 80ms 2024-04-25T21:14:18.614 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:14:18.616 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:14:18.616 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-25T21:14:18.616 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 +490us[ +511us] +/- 66ms 2024-04-25T21:14:18.616 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -1181us[-1161us] +/- 72ms 2024-04-25T21:14:18.616 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +727us[ +748us] +/- 80ms 2024-04-25T21:14:18.616 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:14:18.616 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:14:18.616 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-25T21:14:18.617 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +480us[ +502us] +/- 66ms 2024-04-25T21:14:18.617 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1182us[-1160us] +/- 72ms 2024-04-25T21:14:18.617 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +733us[ +733us] +/- 80ms 2024-04-25T21:14:18.617 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:14:18.617 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T21:14:18.630 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T21:14:18.631 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T21:14:18.642 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:14:18.741 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:14:18.806 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:14:18.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:14:18.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:14:18.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:14:18.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:14:18.928 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:14:18.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:14:18.955 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T21:14:18.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T21:14:18.977 INFO:teuthology.task.internal:Duration was 1932.400732 seconds 2024-04-25T21:14:18.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T21:14:18.988 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T21:14:18.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:14:18.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:14:18.993 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:14:19.027 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:14:19.029 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:14:19.030 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:14:19.411 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T21:14:19.412 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-04-25T21:14:19.412 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:14:19.475 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-04-25T21:14:19.475 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:14:19.497 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T21:14:19.497 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:14:19.526 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T21:14:19.527 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:14:19.530 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:14:19.540 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:14:19.591 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T21:14:19.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:14:19.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:14:19.605 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:14:20.207 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T21:14:20.222 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T21:14:20.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:14:20.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:14:20.271 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:14:20.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T21:14:20.308 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:14:20.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:14:20.314 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:14:20.331 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-04-25T21:14:20.333 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-04-25T21:14:20.357 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-04-25T21:14:20.374 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:14:20.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:14:20.399 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:14:20.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:14:20.430 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:14:20.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:14:20.445 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T21:14:20.457 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T21:14:20.458 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi047 2024-04-25T21:14:20.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:14:20.509 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi092 2024-04-25T21:14:20.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:14:20.544 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653/remote/smithi186 2024-04-25T21:14:20.544 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:14:20.580 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T21:14:20.581 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:14:20.583 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:14:20.585 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:14:20.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T21:14:20.641 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T21:14:20.641 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T21:14:20.709 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T21:14:20.710 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:14:20.712 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:14:20.714 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:14:20.724 INFO:teuthology.orchestra.run.smithi047.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:14 /home/ubuntu/cephtest 2024-04-25T21:14:20.726 INFO:teuthology.orchestra.run.smithi092.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:14 /home/ubuntu/cephtest 2024-04-25T21:14:20.728 INFO:teuthology.orchestra.run.smithi186.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:14 /home/ubuntu/cephtest 2024-04-25T21:14:20.729 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T21:14:20.777 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T21:14:20.802 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T21:14:20.826 DEBUG:teuthology.nuke:shortname: smithi047 2024-04-25T21:14:20.826 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T21:14:20.845 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:36:32.507325', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGLTAP5SraWWthZaMaHi9MKtXXMliswmWAQwrSWAgNKuEKB4+VTxdfXb8uSZN5fPZzZoI+LHF2xoxXemM4QCJ/8='} 2024-04-25T21:14:20.849 DEBUG:teuthology.nuke:shortname: smithi092 2024-04-25T21:14:20.849 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T21:14:20.872 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:36:32.508597', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMdypklKpkF+hjw/7UuCw/DBjZAoQ/++7VZj7uXlAK+x39lQ/dHVYatCVBYL9aroC0A/3sN3fHAGcPzm/TlgYBw='} 2024-04-25T21:14:20.873 DEBUG:teuthology.nuke:shortname: smithi186 2024-04-25T21:14:20.873 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T21:14:20.903 INFO:teuthology.orchestra.console.smithi047:Power off 2024-04-25T21:14:20.903 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T21:14:20.911 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:36:32.505722', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCejTNE7xCNe5W40Lmyc6UVh8tiwuJcqn0Vk53uxWfqsD/vJDU+o/ZpZVlklEuiozQR9HAz9Pu1eDOxH6cZHgx0='} 2024-04-25T21:14:20.927 DEBUG:teuthology.orchestra.console.smithi047:power off output: Chassis Power Control: Down/Off 2024-04-25T21:14:20.927 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:20.943 INFO:teuthology.orchestra.console.smithi092:Power off 2024-04-25T21:14:20.943 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T21:14:20.961 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-04-25T21:14:20.964 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-04-25T21:14:20.965 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:20.978 INFO:teuthology.orchestra.console.smithi186:Power off 2024-04-25T21:14:20.979 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T21:14:20.988 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-04-25T21:14:21.004 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2024-04-25T21:14:21.004 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:21.027 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-25T21:14:24.962 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:24.989 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:25.028 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:25.078 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-04-25T21:14:25.102 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-04-25T21:14:25.139 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-25T21:14:29.082 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:29.103 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:29.140 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:14:29.192 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off 2024-04-25T21:14:29.212 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-04-25T21:14:29.249 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2024-04-25T21:14:29.293 INFO:teuthology.orchestra.console.smithi047:Power off completed 2024-04-25T21:14:29.313 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-04-25T21:14:29.349 INFO:teuthology.orchestra.console.smithi186:Power off completed 2024-04-25T21:14:29.456 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 1932.4007322788239 failure_reason: 'error during quiesce thrashing: Error quiescing set ''9ebddfb8'': 110 (ETIMEDOUT)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-25T21:14:29.456 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T21:14:29.545 INFO:teuthology.run:FAIL